{"licence":{"name":"CC BY-SA 4.0","spdx":"CC-BY-SA-4.0","url":"https://creativecommons.org/licenses/by-sa/4.0/","attribution":"Atlas, a bilingual technical dictionary (https://cmaintz.github.io/tech-atlas/)"},"id":"ai/few-shot-prompting","url":{"en":"https://cmaintz.github.io/tech-atlas/en/terms/ai/few-shot-prompting/","da":"https://cmaintz.github.io/tech-atlas/da/terms/ai/few-shot-prompting/"},"term":{"en":"Few-shot prompting","da":"Few-shot prompting"},"aka":{"en":[],"da":[]},"domain":["ai"],"cluster":"prompting","layer":"inference","status":"current","era":2020,"summary":{"en":"Showing a language model a handful of worked examples inside the request, so it copies the pattern for the new case.","da":"At vise en sprogmodel en håndfuld løste eksempler i selve forespørgslen, så den efterligner mønstret i det nye tilfælde."},"body":{"formal":{"en":"A way of steering a large language model by placing several input-answer pairs in the prompt before the real input; the model picks up the task from these examples at inference time, with no change to its weights.","da":"En måde at styre en stor sprogmodel på ved at placere flere par af input og svar i prompten før det egentlige input; modellen opfanger opgaven fra eksemplerne under inferensen uden ændringer i sine vægte."},"plain":{"en":"Like showing a new cashier three filled-in receipts before asking them to write the fourth - they copy the layout without a lesson.","da":"Som at vise en ny kassemedarbejder tre udfyldte kvitteringer, før de skal skrive den fjerde - de kopierer opsætningen uden undervisning."},"inPractice":{"en":"A customer service lead at a Danish webshop puts five old return notes, each followed by its reason code, above every new note, and the assistant replies with just a code in the same style.","da":"En kundeserviceleder i en dansk webshop sætter fem gamle returbeskeder, hver efterfulgt af sin årsagskode, over hver ny besked, og assistenten svarer med blot en kode i samme stil."},"whyItMatters":{"en":"It lets a team get a new task working in minutes without fine-tuning, but the examples use up space in the context window and the model may copy their odd habits too closely.","da":"Det lader et team få en ny opgave til at virke på få minutter uden finjustering, men eksemplerne optager plads i kontekstvinduet, og modellen kan efterligne deres særheder for tæt."}},"deepDive":{"en":"Brown et al. (2020) used the GPT-3 paper to define the vocabulary still in use: zero-shot (task description only), one-shot (one demonstration) and few-shot (as many demonstrations as fit in the context window, typically 10 to 100 in their 2,048-token setting), all without gradient updates, and they called the underlying capability in-context learning. Their central result was that few-shot performance improved much more steeply with model size than zero-shot performance, which is why the paper was titled \"Language Models are Few-Shot Learners\".\n\nWhat the model learns from the examples is narrower than it looks. Min et al. (2022) found that replacing the gold labels in demonstrations with random labels barely reduced accuracy on many classification tasks; the examples mainly communicate the label space, the input distribution and the output format rather than the input-label mapping itself. Performance is also sensitive to surface choices. Zhao et al. (2021, \"Calibrate Before Use\") documented majority-label bias (favouring the label most common among the examples), recency bias (favouring the label of the last example) and common-token bias, and Lu et al. (2022) showed that simply reordering the same examples could move accuracy from near state-of-the-art to near chance. Sclar et al. (2023) found differences of up to 76 accuracy points from formatting alone.\n\nGood practice follows from those findings: choose diverse, representative examples that cover edge cases, balance the labels, randomise or test their order, keep formatting identical to the real input, and delimit examples clearly (for instance in tagged blocks) so the model does not confuse them with the live input. Dynamic few-shot selects the examples per request by embedding similarity to the input, a small retrieval step. With instruction-tuned chat models, examples are often most effective for pinning down format, tone and tricky boundary cases, while the instruction carries the task definition. With reasoning models the benefit can be smaller and prescriptive examples may constrain the model's own approach.\n\nThe main trade-offs versus alternatives are cost and leakage. Every example is paid for on every call and consumes context window, although a stable example block at the start of the prompt can be served from a prompt cache. Examples copied from production data can put personal data into every request and into provider logs, and the model may reproduce details from them verbatim. When a task needs hundreds of examples, is run at high volume, or requires consistent behaviour that prompting cannot achieve, fine-tuning on the same examples becomes the better option.","da":"Brown m.fl. (2020) brugte GPT-3-artiklen til at fastlægge det ordforråd, der stadig bruges: zero-shot (kun opgavebeskrivelse), one-shot (ét eksempel) og few-shot (så mange eksempler, der kan være i kontekstvinduet, typisk 10 til 100 i deres opsætning med 2.048 tokens), alle uden gradientopdateringer, og de kaldte den bagvedliggende evne in-context learning. Deres hovedresultat var, at few-shot-ydeevnen voksede langt stejlere med modelstørrelsen end zero-shot-ydeevnen, deraf titlen \"Language Models are Few-Shot Learners\".\n\nDet, modellen lærer af eksemplerne, er snævrere, end det ser ud. Min m.fl. (2022) fandt, at når de korrekte etiketter i eksemplerne blev erstattet med tilfældige, faldt præcisionen knap på mange klassifikationsopgaver; eksemplerne formidler primært etiketrummet, inputfordelingen og outputformatet snarere end selve koblingen mellem input og etiket. Ydeevnen er også følsom over for overfladiske valg. Zhao m.fl. (2021, \"Calibrate Before Use\") dokumenterede majority-label bias (at foretrække den etiket, der er hyppigst blandt eksemplerne), recency bias (at foretrække det sidste eksempels etiket) og common-token bias, og Lu m.fl. (2022) viste, at blot at ændre rækkefølgen af de samme eksempler kunne flytte præcisionen fra tæt på det bedste til tæt på tilfældigt gæt. Sclar m.fl. (2023) fandt forskelle på op til 76 procentpoint i præcision alene på grund af formatering.\n\nGod praksis følger af disse fund: Vælg varierede, repræsentative eksempler, der dækker grænsetilfælde, balancér etiketterne, randomisér eller test rækkefølgen, hold formateringen identisk med det rigtige input, og afgræns eksemplerne tydeligt (fx i mærkede blokke), så modellen ikke forveksler dem med det aktuelle input. Dynamisk few-shot vælger eksemplerne pr. kald ud fra embedding-lighed med inputtet, et lille søgetrin. Med instruktionstilpassede chatmodeller er eksempler ofte mest effektive til at fastlægge format, tone og vanskelige grænsetilfælde, mens instruktionen bærer selve opgavedefinitionen. Med ræsonnementsmodeller kan gevinsten være mindre, og foreskrivende eksempler kan begrænse modellens egen fremgangsmåde.\n\nDe vigtigste afvejninger over for alternativerne er pris og læk. Hvert eksempel betales ved hvert kald og optager plads i kontekstvinduet, selvom en stabil eksempelblok i starten af prompten kan leveres fra en prompt-cache. Eksempler kopieret fra produktionsdata kan lægge personoplysninger ind i hvert kald og i udbyderens logs, og modellen kan gengive detaljer fra dem ordret. Når en opgave kræver hundredvis af eksempler, køres i stor volumen eller kræver en ensartet adfærd, som prompting ikke kan levere, bliver finjustering på de samme eksempler det bedre valg."},"edges":[{"type":"requires","to":"ai/prompt","confidence":"high","strength":"normal"},{"type":"kind-of","to":"ai/prompt-engineering","confidence":"high","strength":"normal"},{"type":"contrasts-with","to":"ai/zero-shot-prompting","why":{"en":"Few-shot shows the model worked examples first; zero-shot gives only the instruction.","da":"Few-shot viser modellen løste eksempler først; zero-shot giver kun instruktionen."},"confidence":"high","strength":"primary"},{"type":"alternative-to","to":"ai/fine-tuning","why":{"en":"Both teach a model a task from examples; few-shot puts them in each request, while fine-tuning bakes them into the weights.","da":"Begge lærer en model en opgave ud fra eksempler; few-shot lægger dem i hver forespørgsel, mens finjustering bager dem ind i vægtene."},"confidence":"high","strength":"primary"}],"depth":4,"sources":[{"title":"Brown et al. (2020), Language Models are Few-Shot Learners","url":"https://arxiv.org/abs/2005.14165","tier":"reference"},{"title":"Jurafsky & Martin, Speech and Language Processing (3rd ed. draft), chapter on large language models","tier":"textbook"}],"draft":true}