---
title: 'In the News: October 17, 2026 (Midday)'
description: 'A Tencent study finds coding-agent reports let a reader recover 52.0% of task state, against 92.7% from the full execution record.'
canonical_url: 'https://darkfactory.dev/news/2026-10-17-midday'
markdown_url: 'https://darkfactory.dev/news/2026-10-17-midday.md'
collection: news
date_published: '2026-10-17T13:05:00-04:00'
date_modified: '2026-10-17T13:05:00-04:00'
---

# In the News: October 17, 2026 (Midday)


A coding agent can finish a repair and still leave out the failures and workarounds a supervisor would need to hear about. A Tencent measurement study puts a number on that gap. A second paper, from ETH Zurich, finds that the skills, tools and harnesses it tested mostly change an agent's token cost, not its speedup.

## 1. Coding-agent reports leave a 40.7-point gap in what a supervisor can recover

**[What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reports](https://arxiv.org/abs/2610.20003)** · Guoyang Xu, Rui Zhang, Xiuyu Wu and Hao Chen, Tencent · arXiv, October 2026

The authors built a measurement they call CTXGAP. For each coding task they ask the same questions about what happened (failed attempts, workarounds, residual risks) of a reader who sees only the agent's accumulated reports and of a reader who sees the full recorded execution. Across 37 SWE-bench Verified tasks and 4,799 valid probes, the agents answered 93.6% of the questions correctly when asked directly, the full-context reader recovered 92.7%, and the report-only reader recovered 52.0%. At the final checkpoint of 30 tasks whose fixes passed the hidden tests, the gap was still 34.8 points. The authors report similar gaps with Claude Code and two other harnesses on 12 shared issues.

Much of the missing information was in the reports but too thin to use. On a sample of 195 gap items, two human annotators put states mentioned without enough supporting detail at 55.4% and 59.2% of the weighted gap, against 42.1% and 33.7% for states never mentioned. Adding a bare mention of an event did not help (a change of -0.002); adding the supporting evidence raised recovery by 0.189. Full terminal scrollback lifted recovery to 73.6% but did not close the gap.

**Why it matters:** A green test run and a confident summary do not tell you what the agent hit along the way. If you review agent work from its report alone, these numbers suggest you are seeing about half of the task state, so a verification loop needs access to the execution record, not only the write-up. The questions and judgments are model-generated by the authors' own pipeline, they chose the 37-task subset rather than sampling it, and we did not read the appendices.

## 2. Across 40 matched setups, interventions changed speedup twice but token cost by up to 3.0 times

**[HPCAgent-Bench: A Framework for Sound Benchmarking of Code-Optimizing Agents, Harnesses, and Skills](https://arxiv.org/abs/2610.20673)** · Yakup Koray Budanaz, Torsten Hoefler and colleagues, ETH Zurich · arXiv, October 2026

HPCAgent-Bench has 689 kernels for testing agents that optimize code for speed. An external judge checks correctness on hidden fuzzed inputs and times every submission, so an agent cannot report its own results. The authors say the guards were needed: with fixed inputs, one Qwen3.8-27B answer scored a 5309x speedup by caching its result, and 20x on fresh inputs.

The paper's main use is changing one part of an agent's setup at a time. The authors report that a skill packet rarely changed speed, and that on Harness20 no harness changed speedup, while OpenHands cut Qwen3.8-27B's billed tokens 2.1x and mini-SWE-agent cut OSS-120B's 3.0x. Posing a kernel as a git repository with an issue, in the SWE-bench format, changed nothing measurable. Removing the score tool lowered speedup significantly in 3 of 8 setups, and Qwen3.8-27B solved 31 kernels instead of 39 on HIP. The abstract's summary is that interventions significantly changed speedup only twice across 40 matched setups, and token cost by up to 3.0x.

**Why it matters:** The practical finding is about feedback. The agent that could check its candidate against a scoring tool did better than the one working blind, and the harness and skill choices mostly moved cost. If you are choosing between harnesses, measure tokens per solved task before you assume a better result. These are the authors' own results on open-weight models, in high-performance computing kernels, so the transfer to ordinary application code is untested. The benchmark code is released on GitHub; we did not run it.