Precision vs Recall

Why they differ

Precision asks how many of the flagged cases were right; recall asks how many of the real cases were found. Raising one usually lowers the other.

Precision

Evaluation & metrics

Of all the cases a model flags as positive, the share that really are; in security terms, how many alerts were real.

Formal

The number of true positives divided by all cases the model marked positive (true positives plus false positives), taken from a confusion matrix for one class.

In plain English

Like a mushroom picker whose basket holds only good mushrooms; every one they chose was safe, even if they walked past plenty of others.

In practice

At a small engineering firm, the office manager who looks after IT sees the phishing filter hold back 200 emails in a month; 180 really are phishing, so its precision is 90%, and she releases the other 20 by hand.

Why it matters

Low precision buries staff in wrong alerts and teaches them to ignore the tool, so it is the number to raise when the cost of a wrong flag is high.

Recall

Evaluation & metrics

Of all the real positive cases, the share a model manages to find; in security terms, how many real attacks were caught.

Formal

The number of true positives divided by all cases that truly belong to the class (true positives plus false negatives), taken from a confusion matrix for one class.

In plain English

Like a fishing net with small holes, which brings up nearly every fish in the lake, along with plenty of weed and old boots.

In practice

A hospital's IT security officer has an outside firm run 50 test attacks, and the anomaly detection tool raises an alert for 40 of them, so its recall is 80%; the 10 it missed are fixed first.

Why it matters

A missed attack or a missed cancer costs far more than an extra check, so recall is the number to raise when missing a real case is the worst outcome.

Shared connections

Classification Confusion matrix F1 score Model evaluation (evals)