Definition
Latency is the elapsed time from a request or event to its response or completion.
Distinguish it from nearby terms
Throughput measures volume per unit time; a system can have high throughput yet poor latency for individual requests.
Check your understanding
Separate time to first token, token generation time, tool waits, queueing, and end-to-end completion.