Machine learning

Arthur Samuel used the phrase in his 1959 paper on a checkers program that improved by playing against itself. The most cited formal definition is Tom Mitchell's (1997): a program learns from experience E with respect to a class of tasks T and performance measure P if its performance at T, as measured by P, improves with E. In modern terms, most machine learning is empirical risk minimisation: choose a model family (linear models, decision trees, neural networks), a loss function that scores errors, and an optimisation procedure that finds the parameters minimising the average loss over the training data, usually with a regularisation term that penalises complexity.

The goal is not low training error but generalisation to unseen data. The theory behind this rests on the assumption that training and future data are drawn independently from the same distribution (i.i.d.). Valiant's PAC framework (1984) and later VC theory give bounds on how much data a model class needs to generalise. In practice the assumption is routinely violated by distribution shift: covariate shift (input mix changes), label or prior shift (base rates change) and concept drift (the relationship itself changes, as when fraudsters adapt). Deployed models therefore need monitoring and periodic retraining, not a one-time sign-off.

The field is usually divided by the kind of feedback available: supervised learning (labelled examples, giving classification and regression), unsupervised learning (structure without labels), self-supervised learning (labels derived from the data itself, the basis of foundation models) and reinforcement learning (reward signals from interaction). Classical methods such as logistic regression, support vector machines and gradient-boosted trees remain the strongest choice for many tabular business problems; deep learning dominates for images, audio and text.

Machine learning overlaps heavily with statistics but differs in emphasis: statistics traditionally targets interpretable parameters and inference about a population, whereas ML prioritises predictive accuracy on held-out data, measured with a strict split into training, validation and test sets. A frequent practical failure is data leakage, where information unavailable at prediction time (a future timestamp, a field filled in after the outcome) slips into the features and inflates offline scores.

Governance consequences follow from the mechanism. Because behaviour is induced from data rather than specified, documentation must cover the data and the training procedure as well as the code: tools such as model cards and datasheets for datasets, and requirements such as Article 10 (data governance) and Article 11 (technical documentation) of the EU AI Act for high-risk systems, exist precisely because the rules cannot be read from source code.