{"licence":{"name":"CC BY-SA 4.0","spdx":"CC-BY-SA-4.0","url":"https://creativecommons.org/licenses/by-sa/4.0/","attribution":"Atlas, a bilingual technical dictionary (https://cmaintz.github.io/tech-atlas/)"},"id":"ai/foundation-model","url":{"en":"https://cmaintz.github.io/tech-atlas/en/terms/ai/foundation-model/","da":"https://cmaintz.github.io/tech-atlas/da/terms/ai/foundation-model/"},"term":{"en":"Foundation model","da":"Grundmodel (foundation model)"},"aka":{"en":[],"da":["foundation model"]},"domain":["ai"],"cluster":"model-architecture","layer":"model","status":"current","era":2021,"summary":{"en":"A large model built once on huge amounts of broad data and then reused as the starting point for many different tasks.","da":"En stor model, der bygges én gang på enorme mængder bred data og derefter genbruges som udgangspunkt for mange forskellige opgaver."},"body":{"formal":{"en":"A deep learning model given pretraining on broad data at scale, usually by self-supervised learning, so that it can be adapted - by fine-tuning, prompts or tools - to a wide range of tasks it was not built for.","da":"En deep learning-model, der har fået fortræning på bred data i stor skala, typisk ved selvsuperviseret læring, så den kan tilpasses - med finjustering, prompts eller værktøjer - til en lang række opgaver, den ikke blev bygget til."},"plain":{"en":"Like a general education that takes years to get, after which a person can learn to be a nurse, a lawyer or a cook in far less time.","da":"Som en almen uddannelse, der tager år at få, hvorefter en person langt hurtigere kan lære at blive sygeplejerske, jurist eller kok."},"inPractice":{"en":"An insurance company's AI lead does not train a model from scratch; she takes a foundation model from a provider and adapts it both to answer customers about their policies and to sum up claim files.","da":"Den AI-ansvarlige i et forsikringsselskab træner ikke en model fra bunden; hun tager en grundmodel fra en udbyder og tilpasser den både til at besvare kundernes spørgsmål om deres police og til at opsummere skadesager."},"whyItMatters":{"en":"Many products rest on the same few foundation models, so a flaw, bias or weakness in one of them spreads to every system built on top of it.","da":"Mange produkter hviler på de samme få grundmodeller, så en fejl, bias eller svaghed i én af dem spreder sig til alle systemer bygget ovenpå."}},"deepDive":{"en":"The term was coined in August 2021 by Bommasani et al. at Stanford's Center for Research on Foundation Models to name a shift in how AI systems are built rather than a new architecture. The report identified two properties: emergence, meaning capabilities that were not explicitly trained for appear as models scale, and homogenisation, meaning that one base model is adapted for many downstream uses, so that its strengths and defects propagate to all of them. Technically a foundation model is almost always a transformer trained with a self-supervised objective, such as next-token prediction, masked-token prediction or contrastive image-text alignment, on web-scale corpora of hundreds of billions to tens of trillions of tokens.\n\nScaling behaviour drove the approach. Kaplan et al. (2020) reported smooth power-law relations between loss and parameters, data and compute; Hoffmann et al. (2022, \"Chinchilla\") revised the compute-optimal ratio to roughly 20 training tokens per parameter, and many later models are deliberately trained far beyond that ratio because a smaller model trained longer is cheaper to serve. Adaptation happens at several layers: prompting and in-context learning with no weight changes, retrieval-augmented generation, parameter-efficient fine-tuning such as LoRA, full fine-tuning, and post-training with instruction tuning and preference optimisation (RLHF, DPO). A released \"base\" model and its \"instruct\" or \"chat\" variant are therefore different artefacts with different behaviour and risk profiles.\n\nIn EU law the relevant category is the general-purpose AI model in Art. 3(63) of the AI Act (Regulation (EU) 2024/1689). Recital 98 states that models with at least a billion parameters trained on large amounts of data with self-supervision at scale should be considered to display significant generality, and the Commission's guidelines use training compute above 10²³ FLOP together with the ability to generate text, audio, images or video as an indicative criterion. Art. 51(2) presumes high-impact capabilities, and hence systemic risk, above 10²⁵ FLOP of cumulative training compute. Provider obligations under Art. 53 (technical documentation, information for downstream providers, a copyright policy and a public summary of training content) and the additional Art. 55 duties for systemic-risk models have applied since 2 August 2025; Art. 53(2) relieves certain open-source releases of the documentation duties, but not if the model has systemic risk.\n\nThe homogenisation point is the main operational risk. A vulnerability class such as a jailbreak technique, a memorised data leak or a systematic bias in the base model is inherited by every fine-tune and product built on it, and a provider's deprecation or silent update of a hosted model changes downstream behaviour without any change on the deployer's side. Pinning model versions, keeping an evaluation suite for each use case and recording which base model and version a system depends on are therefore standard controls.","da":"Begrebet blev skabt i august 2021 af Bommasani m.fl. ved Stanfords Center for Research on Foundation Models for at navngive et skifte i, hvordan AI-systemer bygges, snarere end en ny arkitektur. Rapporten pegede på to egenskaber: emergens, dvs. at evner, der ikke er trænet eksplicit, opstår, når modellerne skaleres, og homogenisering, dvs. at én basismodel tilpasses til mange anvendelser, så dens styrker og fejl breder sig til dem alle. Teknisk er en grundmodel næsten altid en transformer trænet med et selvsuperviseret mål, fx forudsigelse af næste token, forudsigelse af maskerede tokens eller kontrastiv sammenkobling af billede og tekst, på korpusser i internetskala fra hundreder af milliarder til titusinder af milliarder tokens.\n\nSkaleringsadfærden drev tilgangen. Kaplan m.fl. (2020) rapporterede jævne potenslove mellem tab og parametre, data og regnekraft; Hoffmann m.fl. (2022, \"Chinchilla\") justerede det regneoptimale forhold til omtrent 20 træningstokens pr. parameter, og mange senere modeller trænes bevidst langt ud over det forhold, fordi en mindre model trænet længere er billigere at drive. Tilpasning sker i flere lag: prompting og in-context learning uden ændring af vægte, retrieval-augmented generation, parametereffektiv finjustering som LoRA, fuld finjustering og eftertræning med instruktionstræning og præferenceoptimering (RLHF, DPO). En udgivet \"base\"-model og dens \"instruct\"- eller \"chat\"-variant er derfor forskellige artefakter med forskellig adfærd og risikoprofil.\n\nI EU-retten er den relevante kategori AI-modellen til almen brug i AI-forordningens art. 3, nr. 63 (forordning (EU) 2024/1689). Betragtning 98 siger, at modeller med mindst en milliard parametre, trænet på store datamængder med selvsupervision i stor skala, bør anses for at have betydelig generalitet, og Kommissionens retningslinjer bruger træningsberegning over 10²³ FLOP sammen med evnen til at generere tekst, lyd, billeder eller video som vejledende kriterium. Art. 51, stk. 2, formoder kapaciteter med stor virkning, og dermed systemisk risiko, over 10²⁵ FLOP samlet træningsberegning. Udbyderforpligtelserne i art. 53 (teknisk dokumentation, oplysninger til downstream-udbydere, en ophavsretspolitik og et offentligt resumé af træningsindholdet) og de ekstra pligter i art. 55 for modeller med systemisk risiko har gældt siden 2. august 2025; art. 53, stk. 2, fritager visse open source-udgivelser for dokumentationspligterne, men ikke hvis modellen har systemisk risiko.\n\nHomogeniseringen er den største driftsmæssige risiko. En sårbarhedsklasse som en jailbreak-teknik, et memoreret datalæk eller en systematisk bias i basismodellen arves af hver finjustering og hvert produkt bygget ovenpå, og når en udbyder udfaser eller i stilhed opdaterer en hostet model, ændres adfærden hos brugerne uden nogen ændring på deres side. Fastlåsning af modelversioner, en evalueringssuite for hver anvendelse og registrering af, hvilken basismodel og version et system afhænger af, er derfor standardkontroller."},"edges":[{"type":"requires","to":"ai/pretraining","confidence":"high","strength":"normal"},{"type":"requires","to":"ai/self-supervised-learning","confidence":"high","strength":"normal"},{"type":"kind-of","to":"ai/deep-learning","confidence":"high","strength":"normal"},{"type":"contrasts-with","to":"ai/general-purpose-ai-model","why":{"en":"A foundation model is the research name for the idea; a general-purpose AI model is the legal category in the EU AI Act, with its own tests and duties.","da":"En grundmodel er forskningens navn for idéen; en AI-model til almen brug er den juridiske kategori i EU's AI-forordning med egne kriterier og pligter."},"confidence":"high","strength":"primary"},{"type":"used-with","to":"ai/transfer-learning","why":{"en":"A foundation model is useful because what it learned in pretraining carries over to new tasks.","da":"En grundmodel er nyttig, fordi det, den lærte under fortræningen, kan overføres til nye opgaver."},"confidence":"high","strength":"normal"}],"depth":3,"sources":[{"title":"Bommasani et al. (2021), On the Opportunities and Risks of Foundation Models","tier":"reference","publisher":"Stanford CRFM"},{"title":"NIST AI 600-1 - Artificial Intelligence Risk Management Framework - Generative Artificial Intelligence Profile","tier":"standard","publisher":"NIST"},{"title":"General-Purpose AI Models in the AI Act - Questions & Answers","url":"https://digital-strategy.ec.europa.eu/en/faqs/general-purpose-ai-models-ai-act-questions-answers","tier":"official-doc","publisher":"European Commission"}],"draft":true}