Test set vs Training data

Why they differ

Training data is what the model learns from; the test set is kept strictly apart so it can measure the model fairly.

Evaluation & metrics

Examples locked away while a model is built and opened once at the end to give an honest final score.

Formal

A part of the labelled data that plays no role in model training or in choosing settings, used only after all choices are fixed to estimate how well the model will do in real use.

In plain English

Like the sealed envelope of exam questions that stays in the head teacher's safe until exam day, so nobody could have practised on them.

In practice

After weeks of adjusting a model that predicts which patients will miss hospital appointments, a data analyst in a region runs it once on 5,000 held-back appointments, reports that figure to management and does not tweak the model again.

Why it matters

If its examples leak into the training data, or it is checked again and again, the score flatters the model, and a weak system can be put into use on real people.

Training data

Machine learning basics

The examples a model learns from; its behaviour, its blind spots and its mistakes all come from what is in them.

Formal

The collection of examples, often with the correct answers attached, used during model training to set a model's internal numbers; kept apart from the data later used to test it.

In plain English

Like the textbooks and past exams a student studies from. If they are wrong, one-sided or out of date, so is what the student learns.

In practice

A pension fund wants to train a model on ten years of member emails and must first check which of them hold personal data and on what legal basis they may be used.

Why it matters

Training data is an asset to protect and a source of risk. It can leak personal data, carry unfair patterns, or be quietly changed by an attacker.

Shared connections

Model training Validation set