Definition
A unit into which model input or output is segmented for processing; a token may be a whole word, part of a word, punctuation, or another symbol.
Distinguish it from nearby terms
Tokens are model-processing units, not characters or words, and tokenization varies by model.
Check your understanding
Cost and context limits are usually measured in tokens, so the same text can consume different budgets across models.