Anthropic says Claude made claude.ai about 3x faster and ratcheted each win into CI

The team ran the August sprint through Claude Tag (beta), on "an internal research model roughly comparable to Opus 5.5." Claude found bottlenecks, built benchmarks, opened pull requests and watched deploys. Engineers set goals and approved every change. By the post's count, more than 150 threads ran at a time and more than three thousand changes merged "without a single customer-facing incident or rollback." At the 75th percentile, time to a typeable page on a fresh claude.ai load fell from 3.1 seconds to 0.55. Across thirteen measurements the average speedup was 3.1x (geometric mean, real-user monitoring, August 13 against August 27).

Each new benchmark had two jobs: a metric Claude could move in the lab, and a CI guardrail "with a number that could only ratchet down." A daily job lowered each ceiling whenever the count fell. Benchmarks that were flaky or did not track user latency were thrown out, and the team told Claude it would "unship the benches for any candidates that cannot prove" wall-clock wins. On two hot paths, cutting instruction counts by 48% and 31% under Valgrind cut wall-clock time by 78% and 44%. Every PR also went through automated review and at least one human approval, unit tests came before optimizations, and anything user-visible shipped behind a short-lived feature flag. The sprint created nearly two hundred flags and retired more than half of them by the end.

The authors write that "The loop was productive, but it wasn't autonomous." People pushed Claude to take on more scope, ruled on user-visible tradeoffs, and kept each thread narrow. One engineer's instruction to Claude was "please be braver." On Hacker News, where the post had 130 points a little under four hours after it was submitted, much of the reaction was skeptical. Several commenters asked how slow the app had been to begin with, and RomanKornev asked how much less readable the code became after "ratcheting the benchmark down."

Why it matters: Yesterday we ran Max Woolf's account of asking agents to make Rust code faster, and the lesson we drew was that a loop pointed at one number will find the cheapest way to hit it. Anthropic's answer is a gate before the climb: a proxy has to show it tracks what users feel, and then it becomes a CI ceiling so the gain cannot quietly slip back. That rule carries over to any team pointing agents at a metric. The 3x figure is Anthropic's measurement of its own product and cannot be checked from outside.