Fine-tuning

Fine-tuning continues gradient descent from pretrained weights instead of a random initialisation, usually with a much smaller learning rate and dataset than pretraining. For language models the most common form is supervised fine-tuning (SFT): the model is trained with the ordinary next-token cross-entropy loss on prompt-response pairs, with the loss typically masked so that only the response tokens are scored. Preference-based stages such as RLHF or Direct Preference Optimization (Rafailov et al., 2023) are also fine-tuning in the broad sense; instruction tuning is SFT on a large, diverse set of instructions, which is what turns a raw pretrained model into an assistant.

Full fine-tuning updates every weight and needs optimiser state for all of them, which for a large model means several times the memory of the weights alone. Parameter-efficient fine-tuning (PEFT) avoids this. LoRA (Hu et al., 2021) freezes the original weight matrix W and learns a low-rank update BA, where B and A have a small inner rank r (often 8 to 64) and the update is scaled by alpha divided by r; for GPT-3 175B the authors reported roughly 10,000 times fewer trainable parameters. QLoRA (Dettmers et al., 2023) combines LoRA with a 4-bit quantised base model and made fine-tuning a 65-billion-parameter model feasible on a single 48 GB GPU. Adapters can be merged into the weights for serving or kept separate and swapped per tenant.

Fine-tuning is good at changing behaviour - format, tone, domain vocabulary, classification labels, tool-call conventions - and relatively poor at reliably adding new facts, which is why the usual advice is RAG for knowledge and fine-tuning for form. Typical failure modes are overfitting to a small dataset, catastrophic forgetting of general capabilities, and erosion of safety training: Qi et al. (2023) showed that fine-tuning an aligned model on as few as ten adversarial examples, through a commercial fine-tuning API, largely removed its refusal behaviour, and that even benign datasets degraded safety somewhat. Evaluation should therefore include regression and safety test sets, not just the target task.

Two governance points follow. Training data is absorbed into the weights and can be partially extracted by memorisation attacks, and there is no reliable way to delete one person's data from a trained model, so honouring GDPR Art. 17 erasure in practice means retraining or not training on personal data in the first place. Under the EU AI Act, the Commission's July 2025 guidelines on general-purpose AI models treat a downstream modifier as the provider of a new model, as an indicative threshold, only where the modification uses more than one third of the original model's training compute, so ordinary fine-tuning by a deployer normally does not trigger the GPAI provider obligations, though other AI Act roles may still apply.