Inference and generation

Test-time compute

working definition

Definition

Additional computation spent during inference, such as longer deliberation, search, candidate generation, or verification, to improve an outcome.

Distinguish it from nearby terms

Training compute changes a model before deployment; test-time compute spends resources while solving a particular request.

Check your understanding

More compute can amplify metric gaming or repeated error unless the search objective and verifier are sound.