Large language model (LLM)

Almost every current LLM is a decoder-only transformer: a stack of identical blocks, each combining causal (masked) multi-head self-attention with a feed-forward network, residual connections and normalisation, on top of a token-embedding table and under an output projection back to the vocabulary. Modern variants typically use rotary position embeddings, RMSNorm, gated feed-forward layers such as SwiGLU and grouped-query attention; many large models are mixture-of-experts, where a router activates only a few expert feed-forward networks per token, so the parameter count and the compute per token diverge. "Large" has no formal threshold; parameter counts in public models range from about a billion to hundreds of billions.

Training runs in stages. Pretraining minimises next-token cross-entropy over trillions of tokens of web text, code, books and increasingly synthetic data. Scaling-law work (Kaplan et al., 2020; Hoffmann et al., 2022, the Chinchilla paper) showed loss falls predictably as a power law in parameters, data and compute, and that compute-optimal training uses on the order of 20 tokens per parameter, though production models are often trained far beyond that to make inference cheaper. Post-training then applies supervised fine-tuning on instruction-response pairs, preference optimisation (RLHF or DPO) and, for reasoning models, reinforcement learning on verifiable tasks. The result is shipped as weights plus a tokenizer and a chat template.

At inference the model runs one forward pass over the prompt (prefill), then one pass per generated token (decode), reusing a KV cache; prefill is compute-bound, decode memory-bandwidth-bound, which is why output tokens cost more and why time to first token and tokens per second are separate metrics. Capabilities such as in-context learning, tool calling and structured output are behaviours of this same loop shaped by training, not separate modules; the model has no database, no clock and no built-in separation between instructions and data.

Regulatory framing: the EU AI Act does not use the term LLM but regulates general-purpose AI models (Art. 3(63)), with provider obligations in Art. 53 that apply from 2 August 2025, covering technical documentation, information for downstream providers, a copyright policy and a public summary of training content. A model is presumed to have systemic risk when its cumulative training compute exceeds 10^25 floating-point operations (Art. 51(2)), which triggers the additional evaluation, incident-reporting and cybersecurity duties of Art. 55. Organisations that only use an LLM through an API are deployers of whatever AI system they build around it, and their duties depend on that system's use case, not on the model.