{"licence":{"name":"CC BY-SA 4.0","spdx":"CC-BY-SA-4.0","url":"https://creativecommons.org/licenses/by-sa/4.0/","attribution":"Atlas, a bilingual technical dictionary (https://cmaintz.github.io/tech-atlas/)"},"id":"ai/zero-shot-prompting","url":{"en":"https://cmaintz.github.io/tech-atlas/en/terms/ai/zero-shot-prompting/","da":"https://cmaintz.github.io/tech-atlas/da/terms/ai/zero-shot-prompting/"},"term":{"en":"Zero-shot prompting","da":"Zero-shot prompting"},"aka":{"en":[],"da":[]},"domain":["ai"],"cluster":"prompting","layer":"inference","status":"current","era":2020,"summary":{"en":"Asking a language model to do a task from an instruction alone, with no worked examples to copy.","da":"At bede en sprogmodel løse en opgave kun ud fra en beskrivelse af den, uden løste eksempler at efterligne."},"body":{"formal":{"en":"A way of using a large language model in which the prompt describes the task but contains no input-answer examples, so the model must rely only on what it learned during pretraining and instruction tuning.","da":"En måde at bruge en stor sprogmodel på, hvor prompten beskriver opgaven, men ikke indeholder eksempler på input og svar, så modellen udelukkende må trække på det, den lærte under fortræning og instruktionstilpasning."},"plain":{"en":"Like asking an experienced florist for “something for a retirement party” without pointing at any bouquets - you rely on what they already know about such occasions.","da":"Som at bede en erfaren blomsterhandler om “noget til en afskedsreception” uden at pege på nogen buketter - man stoler på det, de allerede ved om den slags."},"inPractice":{"en":"A secretary at a Danish primary school asks a chat assistant to “translate this letter to parents into English and Ukrainian, keeping the headings” and gets usable drafts without giving any sample translations.","da":"En skolesekretær på en folkeskole beder en chatassistent om at “oversætte dette forældrebrev til engelsk og ukrainsk og beholde overskrifterne” og får brugbare udkast uden at give nogen eksempeloversættelser."},"whyItMatters":{"en":"It is how most people use chat assistants and the cheapest place to start, but when the output format or judgement must be exactly right, adding examples usually helps.","da":"Det er sådan, de fleste bruger chatassistenter, og det billigste sted at starte, men når svarformen eller vurderingen skal ramme helt rigtigt, hjælper eksempler som regel."}},"deepDive":{"en":"In the GPT-3 paper (Brown et al., 2020) zero-shot meant giving a pretrained model only a natural-language task description and the input, with no demonstrations and no weight updates, and it was the weakest of the three settings: raw pretrained models often continued the text in unexpected ways rather than performing the task, because nothing in pretraining taught them to treat an instruction as something to carry out. Earlier, and separately, \"zero-shot learning\" in machine learning referred to classifying into categories never seen in training, typically via shared attribute or text embeddings, as in CLIP's zero-shot image classification; the prompting sense borrows the name but not the method.\n\nWhat made zero-shot prompting practical was instruction tuning. FLAN (Wei et al., 2021) fine-tuned a 137-billion-parameter pretrained model on more than 60 NLP datasets rephrased as natural-language instructions and found that it outperformed zero-shot GPT-3 175B on 20 of 25 held-out datasets; T0 (Sanh et al., 2021) reported similar findings with multitask prompted training, and InstructGPT (Ouyang et al., 2022) added reinforcement learning from human feedback. Held-out task clusters were essential to the evaluation, since a task seen during instruction tuning is not truly zero-shot. Every modern chat model is the product of this lineage, so everyday use of assistants is overwhelmingly zero-shot.\n\nZero-shot performance depends heavily on how completely the instruction specifies the task. The model must infer the label set, output format, level of detail and edge-case handling from the words alone, so ambiguous instructions produce inconsistent outputs across runs and inputs. Explicit output schemas or structured output, definitions of each category, stated handling of \"none of the above\", and a role in the system prompt close much of the gap to few-shot. Kojima et al. (2022) showed that zero-shot reasoning also improves markedly with a trigger such as \"Let's think step by step\" (zero-shot chain-of-thought), and reasoning models make this unnecessary by reasoning by default.\n\nChoosing between zero-shot and few-shot is an empirical question. Zero-shot is cheaper per call, avoids leaking example data into prompts and logs, and avoids anchoring the model on idiosyncrasies of particular examples. Few-shot tends to win when the desired format is unusual, the label boundaries are subtle, or the output must match a house style. A sound workflow starts zero-shot, builds a small labelled evaluation set, measures, and adds examples only where the measurements show a benefit. Benchmark results reported as zero-shot should be read with contamination in mind: if the test set appeared in pretraining data, the model is not solving the task from the instruction alone.","da":"I GPT-3-artiklen (Brown m.fl., 2020) betød zero-shot, at en fortrænet model kun fik en opgavebeskrivelse i naturligt sprog og selve inputtet uden eksempler og uden vægtopdateringer, og det var den svageste af de tre opsætninger: Rå fortrænede modeller fortsatte ofte teksten på uventede måder i stedet for at løse opgaven, fordi intet i fortræningen lærte dem at behandle en instruktion som noget, der skal udføres. Tidligere og uafhængigt heraf betød \"zero-shot learning\" i maskinlæring at klassificere i kategorier, der aldrig var set under træningen, typisk via fælles attributter eller tekst-embeddings, som i CLIP's zero-shot-billedklassifikation; promptbetydningen låner navnet, men ikke metoden.\n\nDet, der gjorde zero-shot prompting praktisk, var instruktionstilpasning. FLAN (Wei m.fl., 2021) finjusterede en fortrænet model med 137 milliarder parametre på over 60 NLP-datasæt omformuleret som instruktioner i naturligt sprog og fandt, at den slog zero-shot GPT-3 175B på 20 af 25 tilbageholdte datasæt; T0 (Sanh m.fl., 2021) rapporterede lignende resultater med multitask-træning på prompts, og InstructGPT (Ouyang m.fl., 2022) tilføjede forstærkningslæring fra menneskelig feedback. Tilbageholdte opgaveklynger var afgørende for evalueringen, for en opgave, der er set under instruktionstilpasningen, er ikke reelt zero-shot. Alle moderne chatmodeller stammer fra denne linje, så daglig brug af assistenter er overvejende zero-shot.\n\nZero-shot-ydeevnen afhænger meget af, hvor fuldstændigt instruktionen beskriver opgaven. Modellen skal udlede etiketsæt, outputformat, detaljeniveau og håndtering af grænsetilfælde ud fra ordene alene, så tvetydige instruktioner giver uensartet output på tværs af kørsler og input. Eksplicitte outputskemaer eller struktureret output, definitioner af hver kategori, en angivet håndtering af \"ingen af delene\" og en rolle i systemprompten lukker meget af afstanden til few-shot. Kojima m.fl. (2022) viste, at zero-shot-ræsonnement også forbedres markant med en udløser som \"Let's think step by step\" (zero-shot chain-of-thought), og ræsonnementsmodeller gør det overflødigt, fordi de ræsonnerer som standard.\n\nValget mellem zero-shot og few-shot er et empirisk spørgsmål. Zero-shot er billigere pr. kald, undgår at lække eksempeldata til prompts og logs og undgår, at modellen forankres i særheder ved bestemte eksempler. Few-shot vinder typisk, når det ønskede format er usædvanligt, etiketgrænserne er subtile, eller output skal ramme en bestemt husstil. En sund arbejdsgang starter med zero-shot, bygger et lille mærket evalueringssæt, måler og tilføjer kun eksempler, hvor målingerne viser en gevinst. Benchmarkresultater angivet som zero-shot bør læses med kontaminering for øje: Hvis testsættet fandtes i fortræningsdataene, løser modellen ikke opgaven ud fra instruktionen alene."},"edges":[{"type":"requires","to":"ai/instruction-tuning","confidence":"high","strength":"normal"},{"type":"kind-of","to":"ai/prompt-engineering","why":{"en":"It is the simplest prompting style - instructions only - and the baseline other techniques are compared against.","da":"Det er den enkleste promptstil - kun instruktioner - og det udgangspunkt, andre teknikker sammenlignes med."},"confidence":"medium","strength":"normal"}],"depth":4,"sources":[{"title":"Wei et al. (2021), Finetuned Language Models Are Zero-Shot Learners","url":"https://arxiv.org/abs/2109.01652","tier":"reference"},{"title":"Brown et al. (2020), Language Models are Few-Shot Learners","url":"https://arxiv.org/abs/2005.14165","tier":"reference"}],"draft":true}