Definition
Compute is an umbrella term for the processing work or capacity used by an AI system. It is not one universal unit. Training compute often means the total arithmetic performed over a training run. Inference compute may mean work per token, request, or completed task. Infrastructure teams may instead be discussing available accelerator capacity, throughput, utilization, latency, energy, or money.
Useful measurements include floating-point operations, accelerator-hours, and compute-time products such as petaflop/s-days. These are not interchangeable without assumptions about numerical precision, hardware utilization, sparsity, and which operations are counted. Peak hardware FLOPS describe a capacity ceiling, not the work a model performed.
Compute as a tracked input
Dario Amodei and Danny Hernandez's 2018 OpenAI analysis tracked the compute used in prominent AI training runs. Jared Kaplan and colleagues later measured language-model loss against model size, data, and training compute, showing why a fixed compute budget must be allocated across more than parameter count alone.
Distinguish it from nearby terms
Parameter count describes stored model size. Token count describes the amount of text processed. Hardware count describes available capacity. None of them alone tells you the total computation, elapsed time, or cost of a run.
Check your understanding
Two teams report "100 GPU-days." One used newer accelerators at high utilization and low precision; the other used older hardware at 30 percent utilization. What can you compare directly, and what additional measurements do you need?