Classification

Variants differ in output structure. Binary classification picks one of two classes; multiclass picks exactly one of K; multilabel assigns any subset of labels (an email can be both "invoice" and "urgent"). Hierarchical classification respects a taxonomy, and extreme classification deals with tens of thousands of labels or more. Most modern classifiers output a score vector: a logistic sigmoid for the binary case, a softmax that normalises K logits into a probability distribution for multiclass, and independent sigmoids for multilabel. Training minimises cross-entropy (log loss) between the predicted distribution and the true label.

Model families range from logistic regression, naive Bayes, k-nearest neighbours, support vector machines and decision trees to ensembles such as random forests and gradient-boosted trees (XGBoost, LightGBM), which are strong defaults for tabular data, and neural networks for images, audio and text. A persistent naming trap is that logistic regression is a classification method despite its name: it regresses the log-odds, then thresholds.

The decision threshold is a policy choice, not a model property. The default of 0.5 on a binary score is arbitrary; moving it trades false positives against false negatives along the ROC curve, and the right point depends on the relative costs and the base rate. Evaluation therefore uses the confusion matrix and derived measures such as precision, recall, F1 and ROC-AUC, with precision-recall curves preferred under heavy class imbalance. Under imbalance, a fraud model that always predicts "legitimate" can reach 99.9 percent accuracy and catch nothing. Remedies include class weighting, resampling, cost-sensitive loss and choosing the threshold on a validation set against a business cost matrix.

Scores are not automatically probabilities. Modern neural networks tend to be overconfident (Guo et al., 2017), and calibration methods such as Platt scaling, isotonic regression or temperature scaling are fitted on held-out data so that "0.8" means right about 80 percent of the time; reliability diagrams and expected calibration error measure this. Calibration matters whenever scores drive routing, such as sending uncertain cases to a human.

A closed-set classifier has no "none of the above" option and will confidently assign an out-of-distribution input to some known class. Open-set recognition and out-of-distribution detection add a reject option, and selective classification abstains below a confidence level. Adversarial examples exploit the same geometry by pushing inputs across a decision boundary with small perturbations. Machine-learning classification is also unrelated to information-security data classification, which labels information by confidentiality level.