Temperature

Temperature T divides the logits before the softmax: p_i = exp(z_i / T) / Σ_j exp(z_j / T). At T = 1 the model's distribution is unchanged. As T falls toward 0 the distribution sharpens and approaches a one-hot vector on the argmax, so sampling converges to greedy decoding; implementations special-case T = 0 as greedy rather than dividing by zero. As T grows the distribution flattens toward uniform, and the probability mass in the long tail of implausible tokens rises. The name comes from the Boltzmann distribution in statistical mechanics, and the same knob appears elsewhere in machine learning, for example in knowledge distillation (Hinton et al., 2015), where a raised temperature softens a teacher model's outputs.

Temperature changes relative probabilities without changing their order: the most likely token remains the most likely at every T > 0. It interacts with truncation. In most implementations temperature is applied first and top-k, top-p or min-p afterwards, so a high temperature widens the nucleus that top-p keeps, while a low temperature can shrink it to a single token and make top-p irrelevant. That coupling is why providers advise tuning one of the two and leaving the other at its default; Anthropic's API documentation says to alter temperature or top_p but not both, and some newer models reject requests that set both.

Ranges and defaults differ by provider and change over time, so they should be read from the current API reference. OpenAI's Chat Completions API has documented a range of 0 to 2 with a default of 1; Anthropic's Messages API has documented 0 to 1 with a default of 1. Some reasoning models accept only the default value or ignore the parameter, because their reasoning is tuned for a fixed sampling setup. Values numerically equal across providers are not equivalent, since the underlying models' distributions differ.

Common misconceptions: temperature is not a creativity or accuracy dial as such. Low temperature makes output more consistent, not more correct - a model that is confidently wrong stays wrong, and greedy decoding can fall into repetition loops in long outputs. High temperature increases lexical variety but also the rate of factual slips and incoherence, especially in long generations where one improbable token shifts everything after it. And temperature 0 does not guarantee identical outputs on hosted APIs: batching and floating-point reduction order introduce small numerical differences that can flip near-tied tokens. For evaluation, run each test case several times rather than relying on T = 0 for determinism.