How OpenAI's Performance Factory keeps ChatGPT fast · Okay Zed and Cyrus Forbes, OpenAI · OpenAI Engineering, November 19, 2026
OpenAI's Performance Factory sends a second agent to disprove each diagnosis
When a configured performance alert fires, the Regression Workflow opens a task. Codex agents look for the root cause and propose a fix as a pull request; an engineer reviews and deploys it, and another agent then checks whether performance returned to baseline. Task state is kept in a task service outside any agent's context. A dispatch loop assigns tasks that are ready, and subscriptions wake a task on events such as a review comment or a completed deployment. OpenAI says the workflow tracked more than 1,700 investigations and root caused over 600 cases in just over three months.
The post is direct about what went wrong. "Early on, agents sometimes stopped at the first plausible explanation they found, and some of those diagnoses turned out to be wrong." OpenAI added a separate Codex session that tries to disprove each diagnosis, and an adversarial review of each proposed fix as well. In a manual review of 100 root cause suggestions, it graded 70% as accurate. Fixes have been harder than diagnosis: some proposed changes were "more complicated than necessary or introduce new issues." Changes to skills or models are backtested against past incidents. The investigator cannot use saved memories, the incident writeup or earlier runs, and its transcript is checked afterward for anything that gave the answer away. All of these figures are OpenAI's own, and no outside check of them has been published.
Why it matters: The adversarial reviewer and the controls on backtests are the parts to copy. If an investigation agent can see saved notes or evidence gathered after recovery, a change to its skill can score better on past incidents than it will on the next one.