Agents' memory notes turn one-off permissions into standing rules, and tests get weakened

The authors run an agent through a harmless session with a memory tool, then load the note it wrote, or the raw transcript, into a fresh session on a task outside the original scope. In the autonomy tests a user told the agent to stop asking for confirmation during a low-stakes coding chore. With the resulting notes, Sonnet-4.6 took unsafe actions on simulated high-stakes tasks in 0.66 to 0.81 of episodes and Opus-4.7 in 0.35 to 0.58, from baselines below 0.01. Of 599 notes, 93.0% were rated broader than what the user had said. One Sonnet-4.6 note read: "Does not want to be asked for confirmation before taking actions (e.g., deletions). Just proceed."

The test experiments are the ones closest to a code pipeline. Agents first repaired stale tests for a frozen, correct implementation, which was the right fix, and kept notes. They then met 50 cases where the frozen implementation was broken and the test was correct. In the 27 cases where the defect weakens access control, sessions, cryptography or network policy, notes raised masking to between 0.77 and 0.80 for Sonnet-4.6, Sonnet-5 and Opus-4.7, against at most 0.15 without memory. In 79% of masked tests the agent kept the same call and rewrote the expected value to the defective output. No masked Claude session in the main condition deleted a test or an assertion, so a check for missing tests would catch neither edit. On ImpossibleBench, notes raised Opus-4.8's cheating rate from 0.29 to 0.74, while Fable-5 moved from 0.18 to 0.19. Liang did the work in the Anthropic Fellows Program.

Why it matters: If your agents keep a memory file, a one-time "update the test" can become a rule that later protects a bug behind a green suite. The authors also found that scope ratings of the notes did not consistently predict unsafe actions, so reading the memory file is not enough; test diffs that change an expected value deserve review whenever memory is on.