Hyperparameter vs Model parameter

Why they differ

A hyperparameter is set by people before training; a model parameter is learned by the model during training.

Training & optimisation

A setting a person chooses before model training starts, such as step size or number of rounds, and that the model does not learn itself.

Formal

A value that shapes how model training runs or how large the model is, fixed from outside rather than learned from the training data; it is tuned by trying values and comparing results on a validation set.

In plain English

Like the oven heat and baking time you set before the cake goes in; the cake does not choose them, but they decide how it turns out.

In practice

A data analyst in a municipality runs twenty short training jobs overnight on a model that forecasts demand for home care, each with a different step size and batch size, and keeps the combination that does best on held-back cases.

Why it matters

The same model and data can give a strong or a useless result depending on these settings, and searching them multiplies the cost of training.

Model parameter

Training & optimisation

One of the numbers inside a model that is set by learning from data; their count is how model size is usually stated.

Formal

An internal value of a model, mostly the model weights plus small offset values, that model training adjusts to reduce the loss; a large language model can have billions of them.

In plain English

Like the tiles in a huge mosaic; no single tile means much, but together their colours make the picture, and more tiles allow finer detail.

In practice

An IT buyer in a municipality compares a “7B” and a “70B” model for an internal assistant; the figures are billions of model parameters and give a rough guide to memory needs, running cost and ability.

Why it matters

Parameter count drives how much hardware, energy and money a model needs to train and run, and a larger count is no guarantee of a better result on a given task.

Shared connections

Model training