Definition
Test-time compute is the computation a system spends after a request arrives and before it commits to an answer. It can include longer internal reasoning, parallel candidate generation, search, iterative revision, verifier calls, tool use, or repeated execution against tests. The name comes from machine-learning usage in which "test time" means inference, not a software test suite.
The term is broader than output length or reasoning-token count. A short answer selected from several candidates may use more test-time compute than a long answer produced in one pass. Cost and latency should therefore be measured across the entire workflow, including graders, tools, and discarded candidates.
There is no well-supported single coiner for the broad concept. Charlie Snell and coauthors' 2024 paper put the phrase in the title of an influential study, and OpenAI's o1 release made test-time scaling central to discussion of reasoning models. Those results did not show that more compute helps every problem. The value depends on task difficulty, candidate diversity, the search policy, and whether the verifier can recognize a better answer.
Distinguish it from nearby terms
- Training compute changes model parameters before deployment. Test-time compute spends resources on a particular request with the deployed model.
- A reasoning-token budget is one way to allocate test-time compute. Search, tool calls, and independent verification consume compute too.
- Best-of-N spends compute on parallel candidates and then selects among them. A verification loop can spend the same budget on observing and correcting one trajectory.
Check your understanding
An agent produces eight answers and uses the same model to pick the winner. What evidence would show that the added compute improves correctness rather than confidence or metric gaming?