Definition
Temperature rescales a model's next-token logits before they are converted to probabilities. In the common formulation, each logit is divided by a positive temperature T before softmax. T = 1 leaves the distribution unchanged, values below 1 concentrate probability on the leading candidates, and values above 1 flatten the distribution so lower-ranked candidates are more likely to be sampled.
In common generation APIs, temperature is a sampling control. Positive temperatures do not change which token a simple greedy decoder ranks first, although another search procedure may use the rescaled scores. A low value usually reduces variation when sampling, but it does not make the model more knowledgeable or guarantee repeatable output. Provider behavior at exactly zero also varies: some APIs treat it as greedy decoding, while others approximate the limit or apply additional nondeterministic serving behavior.
Use temperature as an experimental control, not as a dial labeled "creativity." Record the model version, seed support, top-p setting, and other decoding parameters when comparing runs. Changing several sampling controls at once makes the result hard to interpret.
Distinguish it from nearby terms
- Top-p changes which candidates remain eligible by cumulative probability mass. Temperature changes their relative probabilities.
- Calibration temperature scaling is a post-training method for making reported confidence better match observed outcomes. It is not the same operation as choosing a decoding temperature for generated text.
- Greedy decoding selects the highest-probability next token instead of sampling. It can still vary across model versions or serving infrastructure.
Check your understanding
Two model configurations differ in temperature, top-p, and model version. Which variables would you hold fixed before attributing an output change to temperature?