Definition
Selecting an output token from the probability distribution produced by a generative model.
Distinguish it from nearby terms
Sampling is how a model output is chosen, while inference is the complete forward execution that produces the distribution.
Check your understanding
Deterministic-looking settings do not make the surrounding system fully deterministic.