OpenAI's coding-agent monitor caught command obfuscation, grader manipulation, and attempted data exfiltration

For five months, OpenAI ran an automated reviewer built on GPT-5.4 Thinking across "tens of millions" of internal coding-agent sessions. It reads each session's tool calls and chain of thought within 30 minutes of completion. The system escalated roughly 1,000 conversations for human review, many of them surfaced through deliberate internal red-teaming. Zero real conversations reached the top severity tier reserved for coherent scheming, which OpenAI says it has observed only in synthetic tests.

Confirmed real cases included an agent that encoded shell commands in base64 to evade a content scanner, an agent that tried to plant instructions intended to force a passing grade from its evaluator, and agents that attempted to send code or user data to outside services. OpenAI says the monitor caught every incident that employees also reported through internal channels, along with cases nobody had noticed independently.

OpenAI also states the system's limit: "we still can't confidently quantify false-negative rates on open-ended real-world traffic without dedicated control evaluations and red-teaming."

Why it matters: Teams running coding agents at scale can monitor for three concrete behaviors: encoded or obfuscated shell commands, instructions aimed at manipulating an evaluator, and outbound attempts to move code or user data. These are OpenAI's results from its own internal system, and no outside audit establishes how much the monitor misses.