Foundation model

The term was coined in August 2021 by Bommasani et al. at Stanford's Center for Research on Foundation Models to name a shift in how AI systems are built rather than a new architecture. The report identified two properties: emergence, meaning capabilities that were not explicitly trained for appear as models scale, and homogenisation, meaning that one base model is adapted for many downstream uses, so that its strengths and defects propagate to all of them. Technically a foundation model is almost always a transformer trained with a self-supervised objective, such as next-token prediction, masked-token prediction or contrastive image-text alignment, on web-scale corpora of hundreds of billions to tens of trillions of tokens.

Scaling behaviour drove the approach. Kaplan et al. (2020) reported smooth power-law relations between loss and parameters, data and compute; Hoffmann et al. (2022, "Chinchilla") revised the compute-optimal ratio to roughly 20 training tokens per parameter, and many later models are deliberately trained far beyond that ratio because a smaller model trained longer is cheaper to serve. Adaptation happens at several layers: prompting and in-context learning with no weight changes, retrieval-augmented generation, parameter-efficient fine-tuning such as LoRA, full fine-tuning, and post-training with instruction tuning and preference optimisation (RLHF, DPO). A released "base" model and its "instruct" or "chat" variant are therefore different artefacts with different behaviour and risk profiles.

In EU law the relevant category is the general-purpose AI model in Art. 3(63) of the AI Act (Regulation (EU) 2024/1689). Recital 98 states that models with at least a billion parameters trained on large amounts of data with self-supervision at scale should be considered to display significant generality, and the Commission's guidelines use training compute above 10²³ FLOP together with the ability to generate text, audio, images or video as an indicative criterion. Art. 51(2) presumes high-impact capabilities, and hence systemic risk, above 10²⁵ FLOP of cumulative training compute. Provider obligations under Art. 53 (technical documentation, information for downstream providers, a copyright policy and a public summary of training content) and the additional Art. 55 duties for systemic-risk models have applied since 2 August 2025; Art. 53(2) relieves certain open-source releases of the documentation duties, but not if the model has systemic risk.

The homogenisation point is the main operational risk. A vulnerability class such as a jailbreak technique, a memorised data leak or a systematic bias in the base model is inherited by every fine-tune and product built on it, and a provider's deprecation or silent update of a hosted model changes downstream behaviour without any change on the deployer's side. Pinning model versions, keeping an evaluation suite for each use case and recording which base model and version a system depends on are therefore standard controls.