Training data vs Validation set

Why they differ

Training data is what the model learns from; the validation set is held back from it and only checked, to choose settings and when to stop.

Training data

Machine learning basics

The examples a model learns from; its behaviour, its blind spots and its mistakes all come from what is in them.

Formal

The collection of examples, often with the correct answers attached, used during model training to set a model's internal numbers; kept apart from the data later used to test it.

In plain English

Like the textbooks and past exams a student studies from. If they are wrong, one-sided or out of date, so is what the student learns.

In practice

A pension fund wants to train a model on ten years of member emails and must first check which of them hold personal data and on what legal basis they may be used.

Why it matters

Training data is an asset to protect and a source of risk. It can leak personal data, carry unfair patterns, or be quietly changed by an attacker.

Validation set

Evaluation & metrics

Examples held back from training and checked again and again while building a model, to choose its settings and decide when to stop.

Formal

A part of the labelled data kept out of the training data and used during model training to compare hyperparameter choices and pick the best round to stop; because choices are tuned to it, its scores are not a fair final result.

In plain English

Like a cook tasting the sauce again and again while it simmers, adjusting salt and time, which helps steer the cooking but is not the guests' honest verdict on the finished dish.

In practice

A data analyst in a ministry tries five settings for a model that sorts public consultation responses by topic and keeps the one that scores best on 1,500 responses held back from training; the test set stays closed until then.

Why it matters

Without it, settings get chosen by looking at the test set, and the final score quietly stops being honest.

Shared connections

Cross-validation Model training Overfitting Test set