Definition
Reusing computation for repeated prompt prefixes or context blocks to reduce inference latency and cost.
Distinguish it from nearby terms
Caching reuses computation; memory retains task-relevant information; retrieval selects information.
Check your understanding
Cache-friendly context should be stable and ordered, but optimization must not freeze stale or unsafe instructions.