---
title: 'In the News: October 17, 2026 (Extra)'
description: 'FDE-Gym finds the best agent scores 0.56 on 95 customer-delivery tasks and resolves 14, with unstated constraints the main bottleneck.'
canonical_url: 'https://darkfactory.dev/news/2026-10-17-extra'
markdown_url: 'https://darkfactory.dev/news/2026-10-17-extra.md'
collection: news
date_published: '2026-10-17T10:25:00-04:00'
date_modified: '2026-10-17T10:25:00-04:00'
---

# In the News: October 17, 2026 (Extra)


A new benchmark tests whether agents can deliver a working system when the customer's request points at the wrong problem and the real constraints sit with stakeholders. The best configuration resolves 14 of 95 tasks. The authors built the benchmark and report the results themselves, and no independent replication exists yet.

## 1. FDE-Gym: the best agent resolves 14 of 95 customer-delivery tasks, and missing constraints are the main bottleneck

**[FDE-Gym: The Last Mile of AI Delivery](https://arxiv.org/abs/2610.22350)** · Dongxu Yang and colleagues, DeepLethe with university and industry contributors · arXiv, October 2026

The benchmark has 95 tasks drawn from 20 customer scenarios across 18 sectors, each adapted by experts from real delivery projects. Agents work in a copy of the customer's production system and can question scripted stakeholders, but each contact carries a hidden cost. Scoring uses the customer's own business metric, from 0 for the unchanged system to 1 for an expert reference solution. The authors report that building exactly what the sponsor asked for scores at most 0.2.

At the hardest setting, where the agent gets only the sponsor's original request, Claude Opus 5.5 leads with a mean score of 0.56 and resolves 14 tasks (a resolved task scores at least 0.8 with no incident). Claude Sonnet 5.5 scores 0.49, GPT-6 Astra 0.38 and GPT-6.1 Sol 0.37. Five of the thirteen configurations resolve none. When the rules and constraints are handed to the agent, the three leading models score 0.73 to 0.82; when they must discover them, 0.39 to 0.62. The authors also report that leading models obtain 72 to 85 percent of the key facts from stakeholders yet deliver only 0.60 to 0.70 of the reference improvement, and that Claude Haiku 4.5 ends below the do-nothing baseline on 79 of 95 tasks.

**Why it matters:** If these numbers hold, the gap in agent-built software is less about writing the code than about finding the constraints nobody wrote down, which is a spec and verification problem. The harness is open source, so a team can run its own agents against it. We read the abstract, method and main results; the later sections and limitations were not read, and the human baseline is still being collected.