Definition
In-context learning is the observed ability of a model to adapt its behavior from instructions, examples, or patterns supplied in the active context without a parameter update. The adaptation exists in the model's response conditioned on that input. It does not by itself persist after the context is removed.
The 2020 GPT-3 paper Language Models are Few-Shot Learners used the term prominently for large language models. The authors evaluated zero-shot, one-shot, and few-shot tasks through text interaction alone, with no gradient updates or task-specific fine-tuning during those evaluations. The paper established a useful behavioral category, not a claim that the model learns by the same mechanism as a person or a training algorithm.
What changes
The model parameters stay fixed, but the conditional computation changes because the context changes. A demonstration can establish a label mapping, output format, task rule, or local pattern. Removing, reordering, or contradicting that material may change the behavior again.
This makes in-context learning useful for rapid adaptation and also fragile. The examples can contain errors, hidden shortcuts, stale facts, or repetitive traces that push an agent toward the wrong action. Treat the resulting behavior as something to evaluate under the intended context distribution, not as a durable capability acquired once.
Distinguish it from nearby terms
- Few-shot prompting is an input design that supplies several demonstrations. In-context learning is the behavior that those demonstrations may elicit.
- Fine-tuning updates parameters and can change future requests without replaying the same examples.
- Agent memory stores or retrieves information outside the model. Loading that memory into a request may support in-context learning, but storage and adaptation are separate functions.
Check your understanding
A model follows a new classification scheme while three demonstrations are present, then reverts when they are removed. Explain why this supports an in-context learning claim but not a fine-tuning claim.