{"licence":{"name":"CC BY-SA 4.0","spdx":"CC-BY-SA-4.0","url":"https://creativecommons.org/licenses/by-sa/4.0/","attribution":"Atlas, a bilingual technical dictionary (https://cmaintz.github.io/tech-atlas/)"},"id":"ai/neural-network","url":{"en":"https://cmaintz.github.io/tech-atlas/en/terms/ai/neural-network/","da":"https://cmaintz.github.io/tech-atlas/da/terms/ai/neural-network/"},"term":{"en":"Neural network","da":"Neuralt netværk"},"aka":{"en":["artificial neural network","ANN"],"da":["kunstigt neuralt netværk"]},"domain":["ai"],"cluster":"ml-fundamentals","layer":"model","status":"current","era":1958,"summary":{"en":"A model made of many small linked units that each weigh their inputs and pass a number on, loosely inspired by the brain.","da":"En model af mange små forbundne enheder, der hver vægter deres input og sender et tal videre, løst inspireret af hjernen."},"body":{"formal":{"en":"A mathematical model of connected units arranged in layers; each unit multiplies its inputs by learned weights, adds them up and passes the result on, and training adjusts the weights.","da":"En matematisk model af forbundne enheder ordnet i lag; hver enhed ganger sine input med lærte vægte, lægger dem sammen og sender resultatet videre, og træningen justerer vægtene."},"plain":{"en":"Like a huge mixing desk with millions of sliders; training slowly nudges each slider until the sound that comes out is right.","da":"Som et kæmpe lydbord med millioner af skydere; træningen skubber langsomt til hver skyder, indtil lyden, der kommer ud, er rigtig."},"inPractice":{"en":"A municipality's email system passes each incoming message through a neural network that gives a score for how likely it is to be phishing, and holds back messages above a set limit.","da":"En kommunes mailsystem sender hver indgående besked gennem et neuralt netværk, der giver en score for, hvor sandsynligt det er, at den er phishing, og holder beskeder over en fastsat grænse tilbage."},"whyItMatters":{"en":"Its knowledge is spread across millions of numbers rather than readable rules, so it is hard to see why it gave a certain answer.","da":"Dens viden er spredt ud over millioner af tal i stedet for læsbare regler, så det er svært at se, hvorfor den gav et bestemt svar."}},"deepDive":{"en":"Each unit computes y = f(w·x + b): a weighted sum of its inputs plus a bias, passed through a nonlinear activation function f. A layer applies this to many units at once, which reduces to a matrix multiplication followed by an elementwise nonlinearity, and a network composes layers. Without the nonlinearity any stack of layers would collapse into a single linear map. Common activations are the sigmoid and tanh (historically), ReLU, max(0, x), which became standard in the early 2010s because it does not saturate for positive inputs, and smooth variants such as GELU used in transformers.\n\nRosenblatt's perceptron (1958) was a single trainable layer with a threshold output. Minsky and Papert's 1969 analysis showed that a single layer cannot represent functions that are not linearly separable, such as XOR, which dampened interest for over a decade. Multilayer perceptrons with hidden layers remove that limit, and the universal approximation theorems (Cybenko 1989 for sigmoids, Hornik 1991 more generally) show that a single hidden layer of sufficient width can approximate any continuous function on a compact domain to arbitrary precision. The theorem guarantees existence only; it says nothing about how many units are needed or whether training will find the weights.\n\nTraining uses backpropagation, popularised by Rumelhart, Hinton and Williams in 1986, which applies the chain rule to compute the gradient of the loss with respect to every weight in one backward sweep, followed by gradient descent or a variant such as Adam. Practical difficulties include vanishing and exploding gradients in deep stacks, addressed with careful initialisation (Glorot/Xavier 2010, He 2015), normalisation layers and residual connections, and sensitivity to learning rate and batch size.\n\nArchitectures specialise the basic pattern through weight sharing and connectivity: convolutional networks share filters across image positions, recurrent networks and LSTMs (Hochreiter and Schmidhuber 1997) share weights across time steps, and transformers (2017) use attention to let every position weigh every other. Graph neural networks pass messages along edges. Parameter counts range from thousands in embedded classifiers to hundreds of billions in large language models.\n\nThe brain analogy is loose. Artificial units are differentiable arithmetic, not spiking neurons, and backpropagation has no established biological counterpart. The practical consequence for audit is that the learned function is distributed across the weight matrices; explainability methods such as saliency maps, SHAP or probing give partial, approximate views rather than the rules a reviewer might expect.","da":"Hver enhed beregner y = f(w·x + b): en vægtet sum af sine input plus en bias, sendt gennem en ikke-lineær aktiveringsfunktion f. Et lag anvender dette på mange enheder på én gang, hvilket svarer til en matrixmultiplikation efterfulgt af en elementvis ikke-linearitet, og et netværk sammensætter lag. Uden ikke-lineariteten ville enhver stak af lag falde sammen til én lineær afbildning. Gængse aktiveringsfunktioner er sigmoid og tanh (historisk), ReLU, max(0, x), som blev standard i begyndelsen af 2010'erne, fordi den ikke mættes for positive input, og glatte varianter som GELU, der bruges i transformere.\n\nRosenblatts perceptron (1958) var ét trænbart lag med et tærskel-output. Minsky og Paperts analyse fra 1969 viste, at ét lag ikke kan repræsentere funktioner, der ikke er lineært separable, fx XOR, hvilket dæmpede interessen i mere end et årti. Flerlagsperceptroner med skjulte lag fjerner den begrænsning, og de universelle approksimationssætninger (Cybenko 1989 for sigmoider, Hornik 1991 mere generelt) viser, at ét skjult lag med tilstrækkelig bredde kan approksimere enhver kontinuert funktion på et kompakt domæne med vilkårlig præcision. Sætningen garanterer kun eksistens; den siger intet om, hvor mange enheder der kræves, eller om træningen finder vægtene.\n\nTræning sker med backpropagation, gjort udbredt af Rumelhart, Hinton og Williams i 1986, som bruger kædereglen til at beregne tabets gradient med hensyn til hver eneste vægt i ét baglæns gennemløb, efterfulgt af gradientnedstigning eller en variant som Adam. Praktiske vanskeligheder omfatter forsvindende og eksploderende gradienter i dybe stakke, som håndteres med omhyggelig initialisering (Glorot/Xavier 2010, He 2015), normaliseringslag og residualforbindelser, samt følsomhed over for læringsrate og batchstørrelse.\n\nArkitekturer specialiserer grundmønstret gennem vægtdeling og forbindelsesmønstre: Konvolutionelle netværk deler filtre på tværs af billedpositioner, rekurrente netværk og LSTM'er (Hochreiter og Schmidhuber 1997) deler vægte på tværs af tidsskridt, og transformere (2017) bruger attention, så hver position kan vægte alle andre. Grafneurale netværk sender beskeder langs kanter. Antallet af parametre spænder fra tusinder i indlejrede klassifikatorer til hundredvis af milliarder i store sprogmodeller.\n\nHjerneanalogien er løs. Kunstige enheder er differentiabel aritmetik, ikke spikende neuroner, og backpropagation har ingen påvist biologisk modpart. Den praktiske konsekvens for revision er, at den lærte funktion er fordelt over vægtmatricerne; forklaringsmetoder som saliency maps, SHAP eller probing giver delvise, tilnærmede indblik frem for de regler, en revisor måske forventer."},"edges":[{"type":"part-of","to":"ai/machine-learning","confidence":"high","strength":"normal"}],"depth":0,"sources":[{"title":"Goodfellow, Bengio & Courville (2016), Deep Learning","url":"https://www.deeplearningbook.org/","tier":"textbook","publisher":"MIT Press"},{"title":"Rosenblatt (1958), The Perceptron","url":"https://doi.org/10.1037/h0042519","tier":"reference","publisher":"Psychological Review"},{"title":"ISO/IEC 22989:2022, Artificial intelligence concepts and terminology","url":"https://www.iso.org/standard/74296.html","tier":"standard","publisher":"ISO/IEC"}],"draft":true}