Definition
The context window is the token capacity a model can process within one inference request. The input may contain system instructions, conversation, retrieved documents, examples, tool schemas, and tool results. In many systems, the same limit also reserves room for tokens generated in the response, though API accounting rules vary.
The limit is measured in tokens, not words, pages, files, or characters. Tokenization makes the practical capacity vary by language, code, formatting, and data representation. A long tool schema or repeated conversation history consumes the same budget needed for task evidence and output.
Capacity is not reliable use
A model accepting a token does not prove that the token will influence the right output. Nelson Liu and coauthors found that models on their long-context tasks often performed worse when relevant information appeared in the middle of the input than at the beginning or end. The exact behavior changes across models and tasks, but the distinction holds: maximum capacity and effective use need separate tests.
Ordering, instruction conflicts, irrelevant material, and the model's training can all affect how well it uses a long prompt. More context may help when it adds needed evidence, or hurt when it adds noise and competing instructions.
Distinguish it from nearby terms
- Persistent memory stores information across requests. A context window lasts for the current inference unless an application saves and resends its contents.
- Retrieval selects material to place into context. It does not expand the model's context limit.
- A KV cache reuses attention state during generation. It improves inference efficiency but does not make the model reliably understand more than its supported context.
- Context engineering decides what to include, omit, order, and compress within the available window.
Operational significance
Budget the window explicitly. Leave room for the expected output and tool results, measure tokens with the model's tokenizer, and test retrieval quality at realistic prompt lengths. When a task exceeds the useful window, split the work around durable state and verification instead of assuming one larger prompt will preserve every dependency.
Check your understanding
A contract clause fits inside a 200,000-token request but is buried among unrelated documents. The capacity claim is satisfied. Whether the model can find and apply the clause is a separate retrieval and evaluation question.