---
title: 'Top-p sampling'
description: 'A decoding method that samples only from the smallest set of candidate tokens whose cumulative probability reaches a chosen threshold.'
canonical_url: 'https://darkfactory.dev/glossary/top-p'
markdown_url: 'https://darkfactory.dev/glossary/top-p.md'
collection: glossary
date_published: '2026-08-03T00:00:00-04:00'
date_modified: '2026-08-26T00:00:00-04:00'
---

# Top-p sampling


## Definition

Top-p sampling sorts candidate tokens from most to least probable, keeps the smallest leading set whose cumulative probability reaches a threshold `p`, renormalizes that set, and samples from it. The number of eligible tokens changes at every generation step. A peaked distribution may need only a few tokens to reach `p`, while a flatter one may need many.

A setting of `p = 1` keeps the full distribution, subject to any other filters. Lower values remove more of the low-probability tail. Top-p does not mean "keep the top p percent of tokens," nor does it keep every token whose individual probability exceeds `p`.

Ari Holtzman and coauthors introduced the method as **nucleus sampling** in their 2019 paper, later published at ICLR 2020. "Top-p" became the common parameter name in generation APIs.

## Distinguish it from nearby terms

- Temperature reshapes relative probabilities. Top-p chooses a variable-sized candidate set based on cumulative mass.
- Top-k sampling always keeps a fixed number of candidates. Top-p keeps as many as the current distribution requires.
- Greedy decoding does not sample from a candidate set; it picks the highest-probability token.

## Check your understanding

A decoding configuration uses `top_p = 0.9`. Does that mean every eligible token has at least 90 percent probability? Explain what the threshold actually controls.

## Also called

nucleus sampling

## Related factory areas

- [Model selection, routing & budgets](https://darkfactory.dev/factory/model-routing-budgets)

## Evidence and further reading

- [Google Machine Learning Glossary](https://developers.google.com/machine-learning/glossary/)
- [The Curious Case of Neural Text Degeneration](https://arxiv.org/abs/1904.09751)
- [Hugging Face: Generation strategies](https://huggingface.co/docs/transformers/generation_strategies)