---
title: Sampling
description: 'Selecting an output token from the probability distribution produced by a generative model.'
canonical_url: 'https://darkfactory.dev/glossary/sampling'
markdown_url: 'https://darkfactory.dev/glossary/sampling.md'
collection: glossary
date_published: '2026-08-03T00:00:00-04:00'
date_modified: '2026-08-26T00:00:00-04:00'
---

# Sampling


## Definition

Sampling is a decoding strategy that selects the next token by drawing from a probability distribution rather than always taking the single highest-probability token. The chosen token is appended to the sequence, the model produces a new distribution for the next position, and the process repeats until generation stops.

Generation systems often transform the distribution before drawing. Temperature changes how concentrated it is. Top-k keeps only a fixed number of candidates. Top-p, also called nucleus sampling, keeps the smallest set whose cumulative probability reaches a threshold. These controls alter which continuations are reachable and how often they are selected; they do not repair an inaccurate model or guarantee creativity.

## Origin and usage

Random sampling from probability distributions is much older than language models. Holtzman and colleagues introduced nucleus sampling in a 2019 paper, published at ICLR 2020, after showing that maximum-likelihood decoding could produce bland or repetitive text. Their contribution was the dynamic top-p candidate set, not the general idea of sampling.

## Operational significance

Sampling can produce different answers from the same prompt and model. A fixed seed may improve repeatability within a particular stack, but server changes, parallel execution, floating-point behavior, model snapshots, or tool results can still change an end-to-end run. Treat important outputs as probabilistic and test distributions of behavior across repeated trials.

## Distinguish it from nearby terms

- Decoding is the broader process of turning model scores into a sequence. Sampling is one decoding family; greedy decoding and beam search need not sample randomly.
- Greedy decoding chooses the highest-scoring token at each step.
- Beam search keeps several high-scoring partial sequences.
- Inference is the complete model execution that produces scores and generated tokens, rather than only the choice rule.

## Check your understanding

A team sets top-p to 0.9 and calls the system "10 percent random." Explain why that interpretation is wrong and what the threshold actually controls.

## Related terms

- [Logit](https://darkfactory.dev/glossary/logit)
- [Greedy decoding](https://darkfactory.dev/glossary/greedy-decoding)
- [Beam search](https://darkfactory.dev/glossary/beam-search)

## Related factory areas

- [Model selection, routing & budgets](https://darkfactory.dev/factory/model-routing-budgets)

## Evidence and further reading

- [Google Machine Learning Glossary](https://developers.google.com/machine-learning/glossary/)
- [Hugging Face: Generation strategies](https://huggingface.co/docs/transformers/generation_strategies)
- [The Curious Case of Neural Text Degeneration](https://arxiv.org/abs/1904.09751)