{"licence":{"name":"CC BY-SA 4.0","spdx":"CC-BY-SA-4.0","url":"https://creativecommons.org/licenses/by-sa/4.0/","attribution":"Atlas, a bilingual technical dictionary (https://cmaintz.github.io/tech-atlas/)"},"id":"ai/next-token-prediction","url":{"en":"https://cmaintz.github.io/tech-atlas/en/terms/ai/next-token-prediction/","da":"https://cmaintz.github.io/tech-atlas/da/terms/ai/next-token-prediction/"},"term":{"en":"Next-token prediction","da":"Forudsigelse af næste token"},"aka":{"en":["next-word prediction"],"da":["next-token prediction"]},"domain":["ai"],"cluster":"llm","layer":"model","status":"current","summary":{"en":"How a language model writes - it guesses the single most fitting next piece of text, adds it, and repeats until the answer is done.","da":"Sådan skriver en sprogmodel - den gætter det bedst passende næste stykke tekst, tilføjer det og gentager, til svaret er færdigt."},"body":{"formal":{"en":"The task at the heart of a large language model - from all tokens so far, work out how likely each possible next token is; during inference one is picked, added to the text, and the step repeats.","da":"Kerneopgaven i en stor sprogmodel - ud fra alle tokens indtil nu at regne ud, hvor sandsynligt hvert muligt næste token er; under inferens vælges ét, føjes til teksten, og trinnet gentages."},"plain":{"en":"Like finishing a well-known saying - hear “the early bird…” and “catches the worm” comes to mind - done over and over, one word at a time, until a whole answer stands.","da":"Som at gøre et kendt ordsprog færdigt - hør “hvad man ikke har i hovedet…”, og “må man have i benene” melder sig - gjort igen og igen, ét ord ad gangen, til et helt svar står der."},"inPractice":{"en":"A medical secretary at a hospital watches a chat assistant's draft letter appear word by word; when she stops it halfway, half a sentence is left, because each token is chosen only after all the ones before it.","da":"En lægesekretær på et hospital ser en chatassistents udkast til et brev dukke op ord for ord; stopper hun den halvvejs, står der en halv sætning, fordi hvert token først vælges, når alle dem før det er skrevet."},"whyItMatters":{"en":"The model aims for text that fits, not text that is checked, and each answer takes one step per token - which explains both how smooth it sounds and what it costs in money and time.","da":"Modellen sigter mod tekst, der passer, ikke tekst, der er tjekket, og hvert svar tager ét trin per token - hvilket forklarer både, hvor glat den lyder, og hvad den koster i penge og tid."}},"deepDive":{"en":"Formally, an autoregressive language model factorises the probability of a sequence as a product of conditionals, p(x1, ..., xn) = p(x1) · p(x2 | x1) · ... · p(xn | x1, ..., xn−1). At each position the network outputs a vector of logits, one per vocabulary entry (tens to hundreds of thousands of them), and a softmax turns these into a probability distribution over the next token. Training minimises the average negative log-likelihood of the actual next token, the cross-entropy loss; perplexity, the exponential of that loss, is the standard intrinsic metric.\n\nTraining is massively parallel because of teacher forcing and the causal mask. The whole ground-truth sequence is fed in at once, and the attention mask prevents position t from seeing positions after t, so one forward pass yields a loss term at every position simultaneously. Generation cannot be parallelised this way: each new token depends on the previous one, so producing n tokens requires n sequential forward passes. A KV cache avoids recomputing attention keys and values for earlier positions, making each step cost roughly proportional to the current length rather than recomputing the whole prefix, but the sequential dependency remains the main source of latency.\n\nSeveral techniques attack that cost without changing the objective. Speculative decoding (Leviathan et al., 2023; Chen et al., 2023) lets a small draft model propose several tokens that the large model verifies in a single pass, accepting them with a rule that preserves the large model's output distribution exactly. Multi-token prediction heads (Gloeckle et al., 2024) train the model to predict several future tokens at once, which can be used for faster decoding. Diffusion-style language models replace left-to-right generation altogether but remain a minority approach.\n\nThe objective explains several documented quirks. Because each step conditions only on what came before, an early error is never revised, only continued; the model can talk itself into a wrong answer, which chain-of-thought and reasoning training partly exploit and partly counter. The \"reversal curse\" (Berglund et al., 2023) showed that models trained on \"A is B\" often fail to answer \"B is A\", a consequence of learning conditional continuations rather than symmetric facts. Next-token prediction is also why a model's confidence is not calibrated truth: a high-probability token is one that fits the pattern, which is the direct link to hallucination. Selecting the actual token from the distribution is a separate step, sampling, controlled by temperature, top-p and related settings.","da":"Formelt faktoriserer en autoregressiv sprogmodel sandsynligheden for en sekvens som et produkt af betingede sandsynligheder, p(x1, ..., xn) = p(x1) · p(x2 | x1) · ... · p(xn | x1, ..., xn−1). For hver position giver netværket en vektor af logits, én pr. post i ordforrådet (titusinder til hundredtusinder), og en softmax omsætter dem til en sandsynlighedsfordeling over næste token. Træningen minimerer den gennemsnitlige negative log-likelihood for det faktiske næste token, cross-entropy-tabet; perplexity, eksponentialfunktionen af tabet, er det gængse indre mål.\n\nTræningen kan paralleliseres massivt på grund af teacher forcing og den kausale maske. Hele den korrekte sekvens gives ind på én gang, og attention-masken forhindrer position t i at se positionerne efter t, så ét forward pass giver et tabsbidrag for alle positioner samtidig. Generering kan ikke paralleliseres på samme måde: Hvert nyt token afhænger af det forrige, så n tokens kræver n forward passes efter hinanden. En KV-cache undgår at genberegne keys og values for tidligere positioner, så hvert trin koster nogenlunde i forhold til den aktuelle længde i stedet for at genberegne hele præfikset, men den sekventielle afhængighed er stadig hovedkilden til ventetid.\n\nFlere teknikker angriber den omkostning uden at ændre målet. Spekulativ afkodning (Leviathan m.fl., 2023; Chen m.fl., 2023) lader en lille udkastmodel foreslå flere tokens, som den store model verificerer i ét pass, og accepterer dem efter en regel, der bevarer den store models outputfordeling præcist. Multi-token-prediction-hoveder (Gloeckle m.fl., 2024) træner modellen til at forudsige flere kommende tokens på én gang, hvilket kan udnyttes til hurtigere afkodning. Diffusionsbaserede sprogmodeller erstatter generering fra venstre mod højre helt, men er stadig en mindre udbredt tilgang.\n\nMålet forklarer flere dokumenterede særheder. Fordi hvert trin kun betinger på det foregående, bliver en tidlig fejl aldrig rettet, kun videreført; modellen kan tale sig selv ind i et forkert svar, hvilket tankekæder og ræsonnementstræning dels udnytter, dels modvirker. \"Reversal curse\" (Berglund m.fl., 2023) viste, at modeller trænet på \"A er B\" ofte ikke kan svare på \"B er A\", fordi de lærer betingede fortsættelser frem for symmetriske fakta. Forudsigelse af næste token er også grunden til, at modellens sikkerhed ikke er kalibreret sandhed: Et token med høj sandsynlighed er et, der passer til mønstret, og det er den direkte forbindelse til hallucination. At vælge det faktiske token fra fordelingen er et separat trin, sampling, styret af temperatur, top-p og beslægtede indstillinger."},"edges":[{"type":"requires","to":"ai/token","confidence":"high","strength":"normal"},{"type":"requires","to":"ai/inference","confidence":"high","strength":"normal"},{"type":"part-of","to":"ai/large-language-model","confidence":"high","strength":"normal"},{"type":"causes","to":"ai/hallucination","why":{"en":"Each step picks a piece of text that fits well, with no check against facts, so a smooth but wrong answer can grow one likely piece at a time.","da":"Hvert trin vælger et stykke tekst, der passer godt, uden tjek mod fakta, så et glat men forkert svar kan vokse frem ét sandsynligt stykke ad gangen."},"confidence":"high","strength":"primary"},{"type":"used-with","to":"ai/self-supervised-learning","why":{"en":"Guessing the next token and checking it against the real text is also how the model learns - the text supplies its own right answers.","da":"At gætte næste token og tjekke det mod den rigtige tekst er også sådan, modellen lærer - teksten leverer selv de rigtige svar."},"confidence":"high","strength":"primary"},{"type":"used-with","to":"ai/sampling","confidence":"high","strength":"normal"}],"depth":1,"sources":[{"title":"Radford et al. (2019), Language Models are Unsupervised Multitask Learners","tier":"reference"},{"title":"Jurafsky & Martin, Speech and Language Processing","tier":"textbook"}],"draft":true}