Anthropic: fixed memory-write rules cost strong models most of their memory gain

Anthropic describes memory as notes Claude keeps between sessions, and reports several of its own evaluations. On Continual Learning Bench (CL-Bench) 1.0, telling Claude only that a memory folder existed was enough guidance for Claude Opus 4.7 and Opus 5 to improve. When Anthropic added the rule "record only failures", both models lost much of that gain. Claude Sonnet 4.6 behaved differently: prompting it to read its memory files before answering and to test its own theories nearly doubled its score, and Anthropic says Sonnet 5 reads memory before answering 9 times out of 10 without being asked.

The post also covers "dreaming", background passes in which Claude checks its notes against session transcripts and reorganizes them. Anthropic reports that scaling a wiki-building run over a corpus of about 350,000 internal files (roughly 420M tokens) from 2 sessions to 128 sessions raised the score on 25 questions from 13% to 48% of graded criteria. The wiki alone topped out at 48%. Search plus the wiki rose from 54% to 61% as the wiki grew, against 53% for search alone, and saved about 20,000 tokens per question. On a contract-terms task, one shared table written by several agents kept 41 of 57 graded facts; one page per contract, quoting terms word for word, kept 57 of 57.

Two cautions come from Anthropic's own text. The wiki showed no clear gain on a task where criteria could be met from a single document, and on another it grew larger than the source files. In the Pokémon Red comparison, a Sonnet 3.7 run wrote 413 memory files and earned one badge while a Claude Fable 5 run wrote nine files plus a single global.md and earned four; that is one run per model, as Anthropic describes it. The figures are Anthropic's own and the evaluation setups were not independently checked.

Why it matters: If you write memory instructions into a harness, this is evidence against prescribing what the agent may record, at least for current frontier models. It also points to two checks worth testing: a periodic pass that tests notes against transcripts, and a small always-loaded index. Anthropic's own phrasing is that "the top layer of the memory hierarchy has to stay relatively small."