Evaluation and reliability

Emergent behavior

contested definition
Machine-readable Download Markdown

Definition

Emergent behavior is a capability or pattern that appears qualitatively new at a larger scale or higher level of system interaction rather than as an obvious continuation of smaller-scale measurements.

Wei and coauthors used "emergent abilities" in 2022 for language-model abilities that appeared unpredictably at larger scale. Schaeffer and coauthors later showed that some apparent jumps can be created by discontinuous metrics applied to smoothly improving underlying performance. Both observations matter: the measured threshold may be real while the claim of a sudden new capability is not.

System-level emergence can also arise from interaction. Tools, memory, multiple agents, and feedback loops can produce behavior no component displays alone. That does not make the result mysterious. Trace the mechanism, test smaller configurations, and separate the observation from the explanation.

Why it is contested

The disagreement is whether an apparent discontinuity belongs to the system or to the way it was measured. A thresholded benchmark can turn smooth improvement into a sudden score jump, while interactions among tools, memory, and agents can produce genuinely new system-level behavior. The observation and its explanation are separate claims. Calling a capability emergent should identify the scale change, metric, baseline, and evidence that rules out a measurement artifact.

Distinguish it from nearby terms

An emergent ability is an observed performance pattern. Generalization is successful behavior beyond the training examples. A capability may generalize without appearing suddenly, and an apparent jump may come from the metric rather than a new internal mechanism.

Check your understanding

Exact-match accuracy jumps from 0 to 1 when a model's nearly correct answer finally matches formatting. The reported ability looks discontinuous even though token-level quality improved smoothly.