{"licence":{"name":"CC BY-SA 4.0","spdx":"CC-BY-SA-4.0","url":"https://creativecommons.org/licenses/by-sa/4.0/","attribution":"Atlas, a bilingual technical dictionary (https://cmaintz.github.io/tech-atlas/)"},"id":"ai/decoder","url":{"en":"https://cmaintz.github.io/tech-atlas/en/terms/ai/decoder/","da":"https://cmaintz.github.io/tech-atlas/da/terms/ai/decoder/"},"term":{"en":"Decoder","da":"Decoder"},"aka":{"en":[],"da":[]},"domain":["ai"],"cluster":"model-architecture","layer":"model","status":"current","summary":{"en":"The half of a model that writes output one token at a time, each new token based only on what came before it.","da":"Den halvdel af en model, der skriver output ét token ad gangen, hvor hvert nyt token kun bygger på det, der kom før."},"body":{"formal":{"en":"A stack of neural network layers in which each token may look only at earlier tokens, so the stack can guess the next token, add it to the text and repeat; decoder-only designs drop the encoder entirely.","da":"En stak af lag i et neuralt netværk, hvor hvert token kun må kigge på tidligere tokens, så stakken kan gætte næste token, føje det til teksten og gentage; rene decoder-designs dropper encoderen helt."},"plain":{"en":"Like a storyteller by a campfire who can hear everything said so far but never peeks ahead, adding one word, then the next.","da":"Som en historiefortæller ved lejrbålet, der kan høre alt, hvad der er sagt indtil nu, men aldrig kigger frem, og lægger ét ord til ad gangen."},"inPractice":{"en":"A product owner at a webshop sees that product texts twice as long take about twice as long to appear and cost twice as much, because the decoder must produce every token in turn.","da":"En produktejer i en webshop ser, at produkttekster, der er dobbelt så lange, tager omtrent dobbelt så lang tid at lave og koster det dobbelte, fordi decoderen skal frembringe hvert token for sig."},"whyItMatters":{"en":"Almost every large language model is decoder-only, which is why replies arrive gradually and why the length of an answer, not just the question, drives its cost.","da":"Næsten alle store sprogmodeller er rene decodere, og derfor kommer svar gradvist, og derfor er det svarets længde, ikke kun spørgsmålets, der driver prisen."}},"deepDive":{"en":"In the original transformer (Vaswani et al., 2017) each decoder block had three sublayers: masked multi-head self-attention over the tokens generated so far, cross-attention whose queries come from the decoder and whose keys and values come from the encoder output, and a position-wise feed-forward network, each wrapped in a residual connection and layer normalisation. A decoder-only model, introduced at scale by GPT (Radford et al., 2018, a 12-layer stack of about 117 million parameters), simply drops the cross-attention sublayer; the prompt and the continuation live in one sequence and are processed by the same causal stack.\n\nThe causal mask is what makes the stack autoregressive: attention scores for positions j > i are set to −∞ before the softmax, so the representation at position i depends only on tokens 1…i. That allows efficient training with teacher forcing, where all positions of a training sequence are predicted in parallel and the loss is the summed cross-entropy of each next token. At inference the model produces a probability distribution over the vocabulary for the next position, a decoding rule selects a token (greedy argmax, beam search, or sampling with temperature, top-k or top-p/nucleus sampling), the token is appended and the step repeats until an end-of-sequence token or a length limit.\n\nInference therefore has two phases with different bottlenecks. Prefill processes the whole prompt in one parallel pass and is compute-bound; it largely determines time to first token. Decode generates one token per forward pass and is bound by memory bandwidth, because all weights and the growing KV cache must be read for every token; total latency is roughly time to first token plus output length times time per output token. This is why providers usually price output tokens higher than input tokens. Speculative decoding (Leviathan et al.; Chen et al., 2023) lets a small draft model propose several tokens that the large model verifies in one pass, with an acceptance rule that preserves the large model's output distribution.\n\nDecoder-only designs dominate large language models because a single objective, next-token prediction on raw text, scales well and one architecture covers both understanding and generation. Encoder-decoder models such as T5 and BART remain common where input and output are clearly separate, such as translation, speech recognition (Whisper) and some summarisation. Note that \"decoder\" is also used for a different thing in autoencoders and in latent diffusion, where it maps a latent code back to pixels; that decoder is not autoregressive.","da":"I den oprindelige transformer (Vaswani m.fl., 2017) havde hver decoderblok tre dellag: maskeret multi-head self-attention over de tokens, der er genereret indtil nu, cross-attention, hvor queries kommer fra decoderen og keys og values fra encoderens output, og et positionsvist feed-forward-netværk, hvert omgivet af en residualforbindelse og lagnormalisering. En ren decoder-model, som GPT (Radford m.fl., 2018, en stak på 12 lag med omkring 117 millioner parametre) gjorde udbredt, dropper blot cross-attention-dellaget; prompten og fortsættelsen ligger i én sekvens og behandles af den samme kausale stak.\n\nDen kausale maske er det, der gør stakken autoregressiv: Attention-scorer for positioner j > i sættes til −∞ før softmax, så repræsentationen på position i kun afhænger af token 1…i. Det giver effektiv træning med teacher forcing, hvor alle positioner i en træningssekvens forudsiges parallelt, og tabet er den summerede krydsentropi for hvert næste token. Ved inferens giver modellen en sandsynlighedsfordeling over ordforrådet for næste position, en afkodningsregel vælger et token (greedy argmax, beam search eller sampling med temperatur, top-k eller top-p/nucleus sampling), tokenet føjes til, og trinnet gentages, indtil der kommer et slut-token eller en længdegrænse.\n\nInferens har derfor to faser med forskellige flaskehalse. Prefill behandler hele prompten i ét parallelt gennemløb og er compute-bound; den bestemmer i høj grad time to first token. Decode genererer ét token pr. forward-gennemløb og er begrænset af hukommelsesbåndbredden, fordi alle vægte og den voksende KV-cache skal læses for hvert token; den samlede ventetid er groft sagt time to first token plus outputlængden gange tiden pr. output-token. Det er grunden til, at udbydere som regel tager mere for output-tokens end for input-tokens. Speculative decoding (Leviathan m.fl.; Chen m.fl., 2023) lader en lille udkastmodel foreslå flere tokens, som den store model efterprøver i ét gennemløb, med en acceptregel, der bevarer den store models outputfordeling.\n\nRene decoder-designs dominerer blandt store sprogmodeller, fordi ét enkelt træningsmål, forudsigelse af næste token på rå tekst, skalerer godt, og én arkitektur dækker både forståelse og generering. Encoder-decoder-modeller som T5 og BART er stadig udbredte, hvor input og output er klart adskilte, fx oversættelse, talegenkendelse (Whisper) og en del opsummering. Bemærk, at \"decoder\" også bruges om noget andet i autoencodere og i latent diffusion, hvor den oversætter en latent kode tilbage til pixels; den decoder er ikke autoregressiv."},"edges":[{"type":"requires","to":"ai/token","confidence":"high","strength":"normal"},{"type":"requires","to":"ai/attention-mechanism","confidence":"high","strength":"normal"},{"type":"part-of","to":"ai/transformer","confidence":"high","strength":"normal"},{"type":"contrasts-with","to":"ai/encoder","why":{"en":"An encoder reads the whole input in both directions to understand it; a decoder only looks back and writes new tokens one by one.","da":"En encoder læser hele inputtet i begge retninger for at forstå det; en decoder kigger kun tilbage og skriver nye tokens ét efter ét."},"confidence":"high","strength":"primary"},{"type":"used-with","to":"ai/large-language-model","why":{"en":"Most large language models, such as the GPT family, are built as a stack of decoders with no encoder.","da":"De fleste store sprogmodeller, som GPT-familien, er bygget som en stak decodere uden encoder."},"confidence":"high","strength":"primary"},{"type":"used-with","to":"ai/next-token-prediction","confidence":"high","strength":"normal"},{"type":"used-with","to":"ai/sampling","confidence":"high","strength":"normal"}],"depth":3,"sources":[{"title":"Vaswani et al. (2017), Attention Is All You Need","tier":"reference"},{"title":"Radford et al. (2018), Improving Language Understanding by Generative Pre-Training","tier":"reference"},{"title":"Jurafsky & Martin, Speech and Language Processing (3rd ed. draft)","tier":"textbook"}],"draft":true}