{"licence":{"name":"CC BY-SA 4.0","spdx":"CC-BY-SA-4.0","url":"https://creativecommons.org/licenses/by-sa/4.0/","attribution":"Atlas, a bilingual technical dictionary (https://cmaintz.github.io/tech-atlas/)"},"id":"ai/confusion-matrix","url":{"en":"https://cmaintz.github.io/tech-atlas/en/terms/ai/confusion-matrix/","da":"https://cmaintz.github.io/tech-atlas/da/terms/ai/confusion-matrix/"},"term":{"en":"Confusion matrix","da":"Forvekslingsmatrix (confusion matrix)"},"aka":{"en":["error matrix"],"da":["confusion matrix","fejlmatrix"]},"domain":["ai"],"cluster":"evaluation","layer":"theory","status":"current","summary":{"en":"A small table that counts, for every true class, how often a model gave each answer, showing exactly which mistakes it makes.","da":"En lille tabel, der for hver sand klasse tæller, hvor ofte en model gav hvert svar - og viser præcis, hvilke fejl den begår."},"body":{"formal":{"en":"A grid of counts for a classification task with one row per true class and one column per predicted class; with two classes its four cells are true positives, false positives, false negatives and true negatives.","da":"En tabel med optællinger for en klassifikationsopgave med én række pr. sand klasse og én kolonne pr. forudsagt klasse; med to klasser er de fire felter sande positive, falske positive, falske negative og sande negative."},"plain":{"en":"Like a sorting office's end-of-day sheet showing how many letters for each town ended up in each town's bag, so you can see which towns keep getting mixed up.","da":"Som postcentralens opgørelse ved dagens slut, der viser, hvor mange breve til hver by der endte i hver bys sæk, så man kan se, hvilke byer der bliver forvekslet."},"inPractice":{"en":"A case officer in a municipality checks the model that sends citizens' emails on to five departments; the confusion matrix shows that emails about housing benefit keep landing with the pensions team.","da":"En sagsbehandler i en kommune tjekker den model, der sender borgernes mails videre til fem afdelinger; forvekslingsmatrixen viser, at mails om boligstøtte gang på gang havner hos pensionsteamet."},"whyItMatters":{"en":"One overall number hides which errors a model makes; this table shows them, and accuracy, precision and recall are all worked out from its cells.","da":"Ét samlet tal skjuler, hvilke fejl en model laver; denne tabel viser dem, og nøjagtighed, præcision og genkaldelse regnes alle ud fra dens felter."}},"deepDive":{"en":"For k classes the confusion matrix C is a k × k table where C[i][j] counts test items whose true class is i and whose predicted class is j, so correct predictions lie on the diagonal. Orientation is a convention, not a law: scikit-learn's confusion_matrix puts true classes in rows and predictions in columns, while some textbooks and tools transpose it, so the axes must be checked before reading a published matrix. Any class c can be reduced to a one-vs-rest 2 × 2 table: TP = C[c][c], FN = row sum minus the diagonal cell, FP = column sum minus the diagonal cell, TN = everything else.\n\nAlmost every classification metric is a function of these cells. Accuracy is the trace divided by N; recall for a class is the diagonal cell over its row sum, precision the diagonal cell over its column sum, specificity TN / (TN + FP). Row-normalising the matrix (scikit-learn normalize='true') turns each row into per-class recall; column-normalising (normalize='pred') gives per-class precision. Multiclass aggregates differ in how they combine the one-vs-rest tables: macro averaging weights every class equally, micro averaging pools the counts (and in single-label multiclass problems equals accuracy), and weighted averaging weights by class support. Cohen's kappa and the Matthews correlation coefficient are also computed directly from the matrix.\n\nA confusion matrix describes one operating point. A model that outputs scores yields a different matrix for every threshold; the ROC curve (Fawcett, 2006) plots TPR against FPR across that family, and the precision-recall curve does the same for precision and recall. Multiplying the matrix element-wise by a cost matrix and summing gives the expected cost of a threshold, which is how cost-sensitive decisions are made explicit. Because precision depends on prevalence while TPR and FPR do not, a matrix measured on a deliberately balanced test set cannot be read as production performance without reweighting to the real class mix.\n\nIn error analysis the off-diagonal hot spots are the useful part: systematic confusion between two classes often signals overlapping definitions in the labelling guideline, ambiguous source data or a taxonomy that should merge or split classes, rather than a model defect. Computing separate matrices per population slice supports fairness checks such as equalised odds (Hardt et al., 2016), which compares TPR and FPR across groups. For multilabel tasks, scikit-learn's multilabel_confusion_matrix returns one 2 × 2 table per label. Small cells are statistically noisy, so rare classes need enough test examples before their row can be trusted.","da":"For k klasser er forvekslingsmatricen C en k × k-tabel, hvor C[i][j] tæller de testeksempler, hvis sande klasse er i, og som blev forudsagt som j, så de korrekte forudsigelser ligger i diagonalen. Orienteringen er en konvention, ikke en lov: scikit-learns confusion_matrix har sande klasser i rækkerne og forudsigelser i kolonnerne, mens nogle lærebøger og værktøjer bytter om, så akserne skal tjekkes, før man læser en offentliggjort matrix. Enhver klasse c kan reduceres til en en-mod-resten-tabel på 2 × 2: TP = C[c][c], FN = rækkesummen minus diagonalfeltet, FP = kolonnesummen minus diagonalfeltet, TN = resten.\n\nNæsten alle klassifikationsmål er funktioner af disse felter. Nøjagtighed er sporet delt med N; genkaldelse for en klasse er diagonalfeltet over rækkesummen, præcision diagonalfeltet over kolonnesummen, specificitet TN / (TN + FP). Normaliseres matricen pr. række (scikit-learn normalize='true'), bliver hver række til genkaldelse pr. klasse; normaliseres den pr. kolonne (normalize='pred'), får man præcision pr. klasse. Multiklasse-gennemsnit adskiller sig ved, hvordan en-mod-resten-tabellerne kombineres: makrogennemsnit vægter alle klasser ens, mikrogennemsnit lægger optællingerne sammen (og er lig nøjagtighed ved multiklasse med én label pr. eksempel), og vægtet gennemsnit vægter efter antal eksempler pr. klasse. Cohens kappa og Matthews-korrelationskoefficienten beregnes også direkte ud fra matricen.\n\nEn forvekslingsmatrix beskriver ét arbejdspunkt. En model, der giver scorer, giver en ny matrix for hver tærskel; ROC-kurven (Fawcett, 2006) viser TPR mod FPR over hele den familie, og præcision-genkaldelse-kurven gør det samme for præcision og genkaldelse. Ganger man matricen feltvis med en omkostningsmatrix og summerer, får man den forventede omkostning ved en tærskel, hvilket gør omkostningsbevidste beslutninger eksplicitte. Da præcision afhænger af forekomsten, mens TPR og FPR ikke gør, kan en matrix målt på et bevidst balanceret testsæt ikke læses som driftsydelse uden omvægtning til den reelle klassefordeling.\n\nI fejlanalyse er de varme felter uden for diagonalen det interessante: systematisk forveksling mellem to klasser skyldes ofte overlappende definitioner i mærkningsvejledningen, tvetydige kildedata eller en taksonomi, hvor klasser bør slås sammen eller deles, snarere end en fejl i modellen. Separate matricer pr. befolkningsgruppe understøtter fairnesstjek som equalised odds (Hardt m.fl., 2016), der sammenligner TPR og FPR på tværs af grupper. Ved multilabel-opgaver returnerer scikit-learns multilabel_confusion_matrix én 2 × 2-tabel pr. label. Små felter er statistisk støjfyldte, så sjældne klasser kræver nok testeksempler, før deres række kan tages for pålydende."},"edges":[{"type":"requires","to":"ai/classification","confidence":"high","strength":"normal"},{"type":"requires","to":"ai/supervised-learning","confidence":"high","strength":"normal"},{"type":"part-of","to":"ai/model-evaluation","confidence":"high","strength":"normal"},{"type":"used-with","to":"security/false-positive","why":{"en":"False positives are one of its four cells, so the table shows how many wrong alarms sit next to the real catches.","da":"Falske positive er et af dens fire felter, så tabellen viser, hvor mange forkerte alarmer der står ved siden af de ægte fangster."},"confidence":"high","strength":"primary"},{"type":"used-with","to":"ai/test-set","why":{"en":"Its counts are normally made on the test set, so the mistakes it shows are ones the model was not trained to avoid.","da":"Optællingerne laves normalt på testsættet, så de viste fejl er nogle, modellen ikke er trænet til at undgå."},"confidence":"high","strength":"normal"}],"depth":2,"sources":[{"title":"Fawcett (2006), An introduction to ROC analysis","url":"https://doi.org/10.1016/j.patrec.2005.10.010","tier":"reference","publisher":"Pattern Recognition Letters"},{"title":"Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft (§4.9, Evaluation: Precision, Recall, F-measure)","url":"https://web.stanford.edu/~jurafsky/slp3/","tier":"textbook"},{"title":"scikit-learn User Guide, Metrics and scoring (classification metrics)","url":"https://scikit-learn.org/stable/modules/model_evaluation.html","tier":"official-doc","publisher":"scikit-learn"},{"title":"Russell & Norvig, Artificial Intelligence: A Modern Approach","url":"https://aima.cs.berkeley.edu/","tier":"textbook","publisher":"Pearson"}],"draft":true}