Feature

In the standard supervised setting each example is a feature vector x in R^d together with a label y, and a dataset is a design matrix with one row per example and one column per feature (Goodfellow et al., Deep Learning, ch. 5). Google's ML glossary defines a feature simply as an input variable used in making predictions. Features are numerical, categorical (a postcode, a product type) or ordinal (a rating from one to five); categorical features must be encoded before most models can use them, for example with one-hot encoding or ordinal encoding, and numeric features are often standardised to zero mean and unit variance so that no single scale dominates distance or gradient computations.

Features are distinct from model parameters: features are properties of the data supplied at inference time, while parameters (weights) are learned during training. They are also distinct from the label, which is the value to be predicted. Statistics calls features independent or explanatory variables, covariates or predictors; the machine learning term is used here.

Feature quality bounds model quality. Irrelevant or redundant features add noise and raise the risk of overfitting, which is why feature selection and dimensionality reduction exist. A feature that encodes information unavailable at prediction time, such as a field filled in only after the outcome is known, causes target leakage and inflated offline scores. Features that act as proxies for protected characteristics (postcode for ethnicity, for instance) can carry bias into decisions even when the protected attribute itself is removed.

In deep learning the raw input (pixels, tokens) is still the feature vector, but the network learns intermediate representations, often also called features or learned features, in its hidden layers. Interpretability work on large models uses the word in this second sense, for directions in activation space that correspond to human-meaningful concepts.