Overfitting vs Underfitting

Why they differ

Overfitting learns the examples too closely and fails on new cases; underfitting has not learned enough to do well even on the examples it saw.

Overfitting

Machine learning basics

When a model learns its training examples by heart, including their noise, and then does badly on new cases it has not seen.

Formal

A failure of model training in which a model fits the training data too closely, so it scores well on that data but poorly on new data; found by testing on data held back from training.

In plain English

Like a student who learns last year's exam answers by heart, word for word, and then fails when the questions are slightly changed.

In practice

A pension fund's fraud model finds almost every false claim among the cases it was trained on, but misses most real fraud once live, because it learned details found only in those old cases.

Why it matters

It makes a model look better on paper than in real use, and a model that has learned examples by heart can later repeat private details from its training data.

Underfitting

Training & optimisation

When a model is too simple or trained too little to catch the pattern, so it scores badly even on the examples it learned from.

Formal

A failure of model training in which the model cannot fit even its training data well, with high loss on the training data and on held-back data alike, usually because it has too few model parameters, too few epochs or too little useful information in its input.

In plain English

Like describing every animal as “has legs”; the rule is too rough to tell a dog from a table, even for the animals you studied.

In practice

The owner of a small Danish garden centre forecasts sales with a straight line; it misses both the spring rush and the Christmas-tree peak even in last year's own figures, so she switches to a richer model.

Why it matters

A model like this is wrong in a steady, predictable way, so it can look stable while giving poor decisions; fixing it means more capacity or more training, not more rules.

Shared connections

Bias-variance trade-off Model training