Training data

Standard practice splits available data into three disjoint sets: the training set used to fit weights, a validation set used to choose hyperparameters and stopping points, and a test set touched once for the final estimate. Splits must respect the structure of the data: grouped splits keep all records from one patient or customer on one side, and time-based splits train on the past and test on the future. Violating this causes leakage and optimistic scores. For large language models the analogous problem is benchmark contamination, where test items appear in the web-scale pretraining corpus, so reported benchmark results partly measure memorisation.

Quality problems are well characterised. Label noise from annotators is estimated with inter-annotator agreement statistics such as Cohen's kappa; sampling bias arises when collection differs from deployment (a model trained on one hospital's scanners); historical bias arises when accurate labels encode past discrimination; and class imbalance means rare, often important cases are underrepresented. Deduplication matters at scale: duplicated sequences in web corpora increase memorisation and the chance that a model reproduces personal data verbatim.

Training data is an attack surface. Data poisoning inserts crafted examples so the model misbehaves in general or only on a trigger (a backdoor). Web-scraped corpora are exposed because an attacker can control content at URLs that will be crawled. Provenance tracking, dataset hashes, access control on labelling pipelines and anomaly checks on new data are the corresponding controls, alongside documentation such as datasheets for datasets.

The legal framework in Europe is layered. Personal data in a training set needs a legal basis under GDPR Article 6, and special categories under Article 9; purpose limitation and data minimisation apply to training as to any processing. For high-risk systems, Article 10 of the EU AI Act requires data governance practices covering design choices, collection, preparation, and examination for possible biases, and Article 10(3) requires training, validation and testing data sets to be relevant, sufficiently representative and, to the best extent possible, free of errors and complete in view of the intended purpose. Article 10(5) allows exceptional processing of special-category data strictly for bias detection and correction, under safeguards. Providers of general-purpose AI models must publish a sufficiently detailed summary of training content (Article 53(1)(d)) and have a policy to comply with EU copyright law (Article 53(1)(c)), including machine-readable opt-outs from text and data mining under Article 4(3) of the DSM Directive (EU) 2019/790.