---
title: 'Prompt caching'
description: 'Reusing model computation for an unchanged prompt prefix or context block across requests to reduce repeated input processing, latency, and cost.'
canonical_url: 'https://darkfactory.dev/glossary/prompt-caching'
markdown_url: 'https://darkfactory.dev/glossary/prompt-caching.md'
collection: glossary
date_published: '2026-08-03T00:00:00-04:00'
date_modified: '2026-08-26T00:00:00-04:00'
---

# Prompt caching


## Definition

Prompt caching reuses model computation for an unchanged prompt prefix or context block across requests. Transformer inference computes attention keys and values for input tokens. When a later request begins with an eligible matching prefix, a provider can reuse cached state instead of processing that entire prefix again. This can reduce input latency and the billed rate for cached tokens, depending on the provider.

Most implementations are sensitive to prefix identity and order. Static system instructions, tool definitions, project context, and shared documents belong before changing conversation content when the provider uses prefix matching. A timestamp, reordered tool schema, model switch, or edit near the beginning can invalidate the matching prefix and everything after it.

Anthropic's 2026 Claude Code account describes organizing prompts from stable to dynamic and monitoring cache-hit rate as an operating metric. Yichao Ji's earlier Manus account similarly treats stable prompt prefixes as a production design constraint. These are first-party system reports, not evidence that one company coined the general phrase.

## Distinguish it from nearby terms

- A **KV cache** usually refers to saved attention state during generation or serving. Prompt caching exposes reuse of eligible prefix state across separate requests.
- A **response cache** returns a previously generated answer for the same key. Prompt caching still runs the model on the uncached suffix and generates a new output.
- A **semantic cache** may reuse an answer for a similar request. Prompt caching generally depends on exact or provider-defined prefix matching.
- **Agent memory** retains task-relevant state for later use. A prompt cache can disappear without the system forgetting the underlying source data.

## Operational significance

Measure hit rate, cached and uncached input tokens, cache-write cost, latency, and cost per successful task. Keep stable prefixes deterministic, but do not preserve stale or unsafe instructions merely to protect a cache hit. Provider rules differ by model, retention period, account, region, minimum length, and pricing, so treat product details as configuration rather than part of the definition.

Caching can also affect data handling. Review provider retention and isolation guarantees, especially when the cached prefix contains confidential instructions or customer data.

## Check your understanding

A system prompt changes one early timestamp on every request. Even though the remaining 50,000 tokens are identical, the early difference can destroy the reusable prefix. Moving volatile state to the end may restore cache hits without changing model behavior.

## Also called

context caching

## Related terms

- [Prompt compression](https://darkfactory.dev/glossary/prompt-compression)
- [KV cache](https://darkfactory.dev/glossary/kv-cache)
- [Input token](https://darkfactory.dev/glossary/input-token)
- [Token burn](https://darkfactory.dev/glossary/token-burn)
- [Token efficiency](https://darkfactory.dev/glossary/token-efficiency)
- [Token minning](https://darkfactory.dev/glossary/token-minning)

## Related factory areas

- [Context, memory, knowledge & skills](https://darkfactory.dev/factory/context-memory-skills)
- [Economics, capacity & factory FinOps](https://darkfactory.dev/factory/economics-finops)

## Evidence and further reading

- [Prompt Caching Is Everything](https://claude.com/blog/lessons-from-building-claude-code-prompt-caching-is-everything)
- [Context Engineering for AI Agents: Lessons from Building Manus](https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus)