--- title: 'In the News: August 1, 2026, Evening' description: "OpenAI found two harness settings tripled GPT-5.6 Sol's ARC-AGI-3 score and cut its output tokens sixfold." canonical_url: 'https://darkfactory.dev/news/2026-08-01-evening' markdown_url: 'https://darkfactory.dev/news/2026-08-01-evening.md' collection: news date_published: '2026-08-01T20:30:00-04:00' date_modified: '2026-08-01T20:30:00-04:00' --- # In the News: August 1, 2026, Evening OpenAI tripled its model's benchmark score by changing two API settings. Claude Code changelog entries documented new containment controls and boundary escapes, while ARC Prize had never verified a harness vendor's near-perfect result. A benchmark number without a harness description is uninterpretable. ## 1. Two API settings tripled a frontier model's score on a benchmark it was failing **[How enabling two settings tripled our scores on the ARC-AGI-3 benchmark](https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/)** · Ilan Bigio and Ted Sanders, OpenAI · openai.com, 29 July 2026 ARC-AGI-3 asks an agent to work out how an unfamiliar 2D game works without being told. GPT-5.6 Sol scored 7.8% on it, and OpenAI went looking for why. OpenAI traced the result to ARC's harness, which discarded the model's private reasoning after every action, so it re-derived the game from scratch each turn. The harness handled context limits with a rolling truncation window at 175,000 characters, so it also lost its record of past moves. Reimplemented on OpenAI's Responses API with reasoning retained and compaction enabled, the same model went from 13.3% to 38.3% on the public task set, with roughly six times fewer output tokens. OpenAI estimates the average human tester at 48%. Bigio and Sanders: "Benchmarks rarely measure AI models in isolation. They also measure less visible choices about API settings, harness design, and prompting." ARC's generic harness is a deliberate design choice, on the reasoning that it makes model shortcomings visible and comparisons fair. OpenAI recommends using the Responses API rather than Chat Completions, retaining reasoning and enabling compaction. **Why it matters:** This is the vendor-side confirmation of what Thomas Schranz measured from the outside this morning, and it is stronger, because it isolates two named settings and moves both score and token count in opposite directions at once. Check whether your own harness drops reasoning between tool calls, because that is a silent, expensive default. Treat any benchmark number without a harness description as uninterpretable. OpenAI has an obvious interest in a finding that its model was underrated and its own API is the fix, and every figure here is its own. ## 2. Six Claude Code changelog entries, most of them about what an agent may do to your machine **[Claude Code CHANGELOG, versions 2.1.215 to 2.1.220](https://github.com/anthropics/claude-code/blob/main/CHANGELOG.md)** · Anthropic · read in full 20:22 EDT, 1 August 2026 The changelog section spanning versions 2.1.215 to 2.1.220 focused largely on containment. The costliest single line is in 2.1.219: Claude Opus 5 became the default Opus model, with a 1M context window and fast mode at $10 and $50 per million tokens. Two new settings appeared: `sandbox.network.strictAllowlist` to refuse non-allowlisted hosts without prompting, and `sandbox.filesystem.disabled` to drop filesystem isolation while keeping network egress control. Three entries addressed agents escaping their own boundaries: worktree-isolated subagents redirecting git into the shared checkout via `git -C` or `GIT_DIR`, background sessions not canonicalizing symlinked working directories "which could let sessions escape their workspace folder," and scheduled-task writes following a symlink at `.claude` to land outside the project. The entries also described a cap of 20 concurrent subagents and a fix for `--max-budget-usd` not stopping background subagents. Version 2.1.217 stopped subagents spawning nested subagents by default, while 2.1.219 set the default depth to 3. *Correction, August 3: our July 30 edition had already recorded version 2.1.220 with a July 25 date, so these were not six releases in roughly 36 hours. This item is a feature summary of entries observed in Anthropic's mutable documentation changelog, not a reliable release timeline. The GitHub changelog has since been rewritten and no longer preserves the same sequence.* **Why it matters:** The observed changelog said `--max-budget-usd` had not bounded background subagents before the fix. If you did not pin `CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH`, the documented fan-out default changed across two entries. The broader lesson survives the chronology correction: write your limits down rather than silently inheriting mutable defaults. ## 3. A harness claimed 99% on ARC-AGI-3. The verified board's best number is 30.2% **[ARC Prize leaderboard](https://arcprize.org/leaderboard)** · ARC Prize · read 20:25 EDT, 1 August 2026 On 16 July an anonymous team published "Frontier Models with Our Harness Achieve ~99% on ARC-AGI-3 Public," reporting 98.98% against a 42.83% Claude Code baseline on the same models. The Dark Factory sweep read that essay end to end on 17 July and parked the claim as unverified. The question closed this morning: the figures are self-reported public-set results computed from the authors' own run artifacts, and ARC Prize has verified none of them. Read tonight, the leaderboard's highest ARC-AGI-3 entry is Claude Opus 5 at High reasoning, 30.2%, dated 24 July, at $1.45 per task. Every other entry sits at 7.8% or below, most under 1%. Opus 5's 30.2% is roughly four times the previous best and the first double-digit figure on that column. **Why it matters:** Item 1 explains exactly how a self-reported harness result gets to be enormous and meaningless at the same time. A benchmark from the harness's author, on the public set, with no independent run, is a claim about the author's harness and not about the models. The verified frontier also moved without an announcement: Opus 5's 30.2% has sat on that board since 24 July.