← In the News

The effort setting you pick is not the reasoning you get

The Inference Gap · Lon Lundgren, previously Microsoft Research and Amazon Research · X, September 18, 2026

Machine-readable Download Markdown

After a perceived drop in Fable 5 quality following Anthropic's July 17 announcement that the model would stay on subscription plans from July 20 onward, Lundgren started reading the minified Claude Code source on July 30 and by August 1 had built what he describes as a "semantically aware proxy that began transparently collecting live data from the wire". The resulting corpus covers 43,261 Fable 5 invocations at xhigh and max effort, with thinking totals quantified for 43,007 of them, plus 7,853 complete assistant turns, across 65 active-usage days between July 1 and September 7, taken from three machines, two subscription accounts, 25 project groups and 213 recorded sessions. Anthropic's own documentation, as quoted in the article, describes xhigh as Claude "always thinks deeply" and max as "always thinks", with no constraint on thinking depth. The measured distribution: "Half of all invocations delivered no more than 123 tokens of uninterrupted sequential thinking. In 39.2% of invocations, the model performed no thinking at all." Comparing July against August five different ways, to stop busy days or large projects dominating the result, the median decline in delivered thinking runs from 21.9% turn-weighted to 50.6% at the invocation level. Lundgren is explicit about what he has not shown. Nothing in the article "proves that a release or announcement caused a nearby change, nor does it establish whether behavior arose from capacity management, policy changes, routing, adaptive effort logic, or some other internal mechanism."

The finding that bears hardest on fleet work is the fragmentation table. As a turn's total thinking grows, the longest single invocation inside that turn accounts for less and less of it: 99.9% of the total at 2,048 turn-level thinking tokens, 26.0% at 32,768, and 12.1% at 131,072. Lundgren's reading is that adding breadth does not buy back depth. "Parallel sampling is not serial depth. A configured ceiling is not realized consumption." Of the 105 turns in the corpus that reached at least 48 invocations, every one contained calls delivered zero thinking tokens. The collection and analysis code is not published. The article says it "is not yet packaged cleanly enough for release", and that the measurements are now defined well enough for other people holding reasoning-token traces to run the same questions against their own data. A Hacker News thread on the September 20 summary of this work stood at 283 points against 187 comments at 16:57 EDT on September 21.

Why it matters: If the reasoning delivered behind an unchanged effort setting moves week to week, a harness result cannot be attributed to the harness. Your evals, your regression triage and your verification loop are measuring the model and the delivered inference regime at the same time, and only one of those appears in your changelog. The move on offer here is a reframe rather than a fix: when quality drops, ask what regime you were served instead of asking whether the model was changed. This is one person's traffic against one vendor, and the cause is not established. The absence of any published figure that would settle it is the part worth carrying.