{"licence":{"name":"CC BY-SA 4.0","spdx":"CC-BY-SA-4.0","url":"https://creativecommons.org/licenses/by-sa/4.0/","attribution":"Atlas, a bilingual technical dictionary (https://cmaintz.github.io/tech-atlas/)"},"id":"ai/explainability","url":{"en":"https://cmaintz.github.io/tech-atlas/en/terms/ai/explainability/","da":"https://cmaintz.github.io/tech-atlas/da/terms/ai/explainability/"},"term":{"en":"Explainability","da":"Forklarlighed"},"aka":{"en":["explainable AI","XAI","interpretability"],"da":["forklarbar AI","XAI"]},"domain":["ai"],"cluster":"ai-risk","layer":"model","status":"current","era":2016,"summary":{"en":"How well people can understand why an AI system reached a particular result.","da":"Hvor godt mennesker kan forstå, hvorfor et AI-system nåede frem til et bestemt resultat."},"body":{"formal":{"en":"The degree to which the reasons behind a model's output can be shown in terms a person can follow, for example which inputs weighed most, so that the result can be checked, challenged or corrected.","da":"I hvilken grad grundene bag en models resultat kan vises på en måde, et menneske kan følge, for eksempel hvilke oplysninger der vejede tungest, så man kan kontrollere resultatet, klage over det eller rette det."},"plain":{"en":"Like a doctor who does not just say \"take these pills\" but tells you what they saw in your tests and why that led to the choice.","da":"Som en læge, der ikke bare siger \"tag de her piller\", men fortæller, hvad hun så i dine prøver, og hvorfor det førte til valget."},"inPractice":{"en":"A citizen whose application for housing support is flagged for rejection by a municipality's AI tool asks why; the municipality can show that missing income papers, not age or nationality, drove the result.","da":"En borger, hvis ansøgning om boligstøtte bliver indstillet til afslag af kommunens AI-værktøj, spørger hvorfor; kommunen kan vise, at manglende indkomstoplysninger og ikke alder eller nationalitet afgjorde resultatet."},"whyItMatters":{"en":"Without it, nobody can spot hidden unfairness, answer an auditor or give an affected person a real reason, so trust and accountability break down.","da":"Uden den kan ingen opdage skjult uretfærdighed, svare en revisor eller give en berørt person en reel begrundelse, og så bryder tillid og ansvarlighed sammen."}},"deepDive":{"en":"The literature separates interpretability - models whose structure a person can inspect directly, such as sparse linear models, shallow decision trees, rule lists or generalised additive models - from post-hoc explainability, where a separate method approximates why an opaque model produced an output. Explanations are further classified as global (how the model behaves overall) or local (why this one prediction), and as model-specific or model-agnostic. Rudin (2019) argued that for high-stakes tabular decisions an interpretable model often matches black-box accuracy, making post-hoc explanation of a black box the weaker choice.\n\nThe main post-hoc families are feature attribution, example-based and counterfactual methods. LIME (Ribeiro et al., 2016) fits a weighted linear surrogate around the instance using perturbed samples. SHAP (Lundberg and Lee, 2017) assigns each feature its Shapley value from cooperative game theory, the unique attribution satisfying efficiency, symmetry, dummy and additivity; KernelSHAP estimates it by sampling and TreeSHAP computes it exactly for tree ensembles. Gradient methods for neural networks include saliency maps, Integrated Gradients (Sundararajan et al., 2017), which integrates gradients along a path from a baseline, and Grad-CAM for convolutional networks. Counterfactual explanations (Wachter et al., 2017) state the smallest change to the input that would flip the outcome - \"had declared income been above X, the application would have been approved\" - which is often the most useful form for an affected person.\n\nFailure modes are well documented. Attributions depend on the chosen baseline or background distribution; correlated features split credit arbitrarily; saliency maps can look plausible while being insensitive to the model's weights (Adebayo et al., 2018, \"Sanity Checks for Saliency Maps\"); and LIME and SHAP can be manipulated so that a biased model appears to rely on innocuous features. For LLMs, a chain-of-thought is generated text and is not guaranteed to be a faithful account of the computation; mechanistic interpretability (circuits, probing, sparse autoencoders over activations) aims at faithful explanations but is still a research field. NIST IR 8312 accordingly lists four principles: explanation, meaningful, explanation accuracy and knowledge limits.\n\nLegally, GDPR Arts. 13(2)(f), 14(2)(g) and 15(1)(h) give data subjects a right to meaningful information about the logic involved in automated decisions under Art. 22. The CJEU held in SCHUFA (C-634/21, 2023) that a credit score can itself be such a decision, and in Dun & Bradstreet Austria (C-203/22, 2025) that the explanation must enable the person to understand the procedure and principles actually applied, without being defeated wholesale by trade-secret claims. The EU AI Act adds Art. 13 (instructions enabling deployers to interpret output), Art. 14 (human oversight) and Art. 86, a right for affected persons to obtain clear and meaningful explanations of certain decisions based on Annex III high-risk systems.","da":"Litteraturen skelner mellem fortolkelighed (interpretability) - modeller, hvis struktur et menneske kan inspicere direkte, som sparsomme lineære modeller, lave beslutningstræer, regellister eller generaliserede additive modeller - og post-hoc-forklarlighed, hvor en separat metode tilnærmer, hvorfor en uigennemsigtig model gav et bestemt output. Forklaringer inddeles desuden i globale (hvordan modellen opfører sig overordnet) og lokale (hvorfor netop denne forudsigelse) og i modelspecifikke eller modelagnostiske. Rudin (2019) argumenterede for, at en fortolkelig model ved højrisikobeslutninger på tabeldata ofte er lige så præcis som en black box, så post-hoc-forklaring af en black box er det svagere valg.\n\nDe vigtigste post-hoc-familier er feature attribution, eksempelbaserede og kontrafaktiske metoder. LIME (Ribeiro et al., 2016) tilpasser en vægtet lineær surrogatmodel omkring det enkelte tilfælde ud fra forstyrrede eksempler. SHAP (Lundberg og Lee, 2017) tildeler hver feature dens Shapley-værdi fra kooperativ spilteori, den eneste fordeling, der opfylder efficiency, symmetry, dummy og additivity; KernelSHAP estimerer den ved sampling, og TreeSHAP beregner den eksakt for træ-ensembler. Gradientmetoder for neurale netværk omfatter saliency maps, Integrated Gradients (Sundararajan et al., 2017), der integrerer gradienter langs en sti fra en baseline, og Grad-CAM for konvolutionelle netværk. Kontrafaktiske forklaringer (Wachter et al., 2017) angiver den mindste ændring af input, der ville vende udfaldet - \"havde den oplyste indkomst været over X, var ansøgningen blevet godkendt\" - hvilket ofte er den mest brugbare form for en berørt person.\n\nFejlkilderne er veldokumenterede. Attributioner afhænger af den valgte baseline eller baggrundsfordeling; korrelerede features deler æren vilkårligt; saliency maps kan se plausible ud og samtidig være ufølsomme over for modellens vægte (Adebayo et al., 2018, \"Sanity Checks for Saliency Maps\"); og LIME og SHAP kan manipuleres, så en biased model ser ud til at bygge på harmløse features. For LLM'er er en chain-of-thought genereret tekst og ikke nødvendigvis en tro gengivelse af beregningen; mekanistisk interpretability (kredsløb, probing, sparse autoencoders på aktiveringer) sigter mod tro forklaringer, men er stadig et forskningsfelt. NIST IR 8312 opstiller derfor fire principper: forklaring, meningsfuldhed, forklaringens nøjagtighed og videnens grænser.\n\nJuridisk giver GDPR art. 13, stk. 2, litra f, art. 14, stk. 2, litra g, og art. 15, stk. 1, litra h, den registrerede ret til meningsfulde oplysninger om logikken i automatiske afgørelser efter art. 22. EU-Domstolen fastslog i SCHUFA (C-634/21, 2023), at en kreditscore i sig selv kan være en sådan afgørelse, og i Dun & Bradstreet Austria (C-203/22, 2025), at forklaringen skal sætte personen i stand til at forstå den fremgangsmåde og de principper, der faktisk er anvendt, uden at hensynet til forretningshemmeligheder generelt kan afskære den. AI-forordningen tilføjer art. 13 (brugsanvisning, der sætter idriftsættere i stand til at fortolke output), art. 14 (menneskeligt tilsyn) og art. 86, en ret for berørte personer til at få en klar og meningsfuld forklaring på visse afgørelser, der bygger på højrisikosystemer efter bilag III."},"edges":[{"type":"requires","to":"ai/machine-learning","confidence":"high","strength":"normal"},{"type":"part-of","to":"ai/ai-governance","why":{"en":"Frameworks such as the NIST AI RMF treat being explainable as one of the qualities governance must secure.","da":"Rammeværker som NIST AI RMF ser forklarlighed som en af de egenskaber, governance skal sikre."},"confidence":"medium","strength":"normal"},{"type":"mitigates","to":"ai/ai-bias","why":{"en":"Seeing which inputs drive a result makes unfair patterns visible so they can be fixed.","da":"Når man kan se, hvilke oplysninger der styrer et resultat, bliver uretfærdige mønstre synlige, så de kan rettes."},"confidence":"medium","strength":"normal"},{"type":"used-with","to":"ai/model-card","why":{"en":"A model card explains what a model does, how it was tested and where it fails, which makes its behaviour easier to understand and question.","da":"Et modelkort forklarer, hvad en model gør, hvordan den er testet, og hvor den fejler, hvilket gør dens adfærd lettere at forstå og efterprøve."},"confidence":"medium","strength":"normal"}],"depth":2,"sources":[{"title":"NIST AI 100-1 - Artificial Intelligence Risk Management Framework (AI RMF 1.0)","url":"https://doi.org/10.6028/NIST.AI.100-1","tier":"standard","publisher":"NIST"},{"title":"NIST IR 8312 - Four Principles of Explainable Artificial Intelligence","url":"https://doi.org/10.6028/NIST.IR.8312","tier":"standard","publisher":"NIST"},{"title":"Regulation (EU) 2024/1689 (AI Act), Articles 13 and 86","url":"https://eur-lex.europa.eu/eli/reg/2024/1689/oj","tier":"standard","publisher":"European Union"}],"draft":true}