Definition
A token budget is an explicit allocation or ceiling for model-token consumption across a stated boundary, such as a request, run, accepted task, user, workflow, team, or billing period. It converts unconstrained inference into a resource the system can plan, attribute, and stop.
A useful budget says which input, output, reasoning, cache-read, cache-write, retry, and subagent usage counts. It also defines warning thresholds, reservation rules, what happens near exhaustion, who may grant an exception, and how the run exits safely. Without those behaviors, the number is an after-the-fact report rather than a control.
Set budgets around decisions
The right boundary depends on the question. Per-request budgets protect latency and cost. Per-run budgets prevent loops from expanding without limit. Per-accepted-task budgets expose failed attempts and retries. Team or monthly budgets support financial planning but can hide inefficient individual workflows.
A fixed ceiling is not always the best policy. A staged budget can release more inference when early evidence shows that the task is difficult and valuable, then stop when added tokens no longer change the result enough to justify their cost.
Distinguish it from nearby terms
Maximum output tokens cap one response. A context window limits what a model can consider at once. A rate limit controls throughput over time. A financial budget caps money. Provider prices and caching make the conversion between tokens and dollars model-specific, so none of these controls is a substitute for the others.
Check your understanding
An agent has used 90 percent of its budget and has not changed strategy after three failed tests. Should the system buy another 100,000 tokens? Only if the run contract permits it and new evidence justifies a different plan. Otherwise stop, preserve state, and escalate with the failure record.