---
title: 'In the News: November 2, 2026 (Midday)'
description: 'A vendor-run benchmark finds that the browser tool an agent uses changes bug-finding results by 10% to 58%, and that even the best agent found 19.3% of labeled bugs.'
canonical_url: 'https://darkfactory.dev/news/2026-11-02-midday'
markdown_url: 'https://darkfactory.dev/news/2026-11-02-midday.md'
collection: news
date_published: '2026-11-02T13:30:00-05:00'
date_modified: '2026-11-02T13:30:00-05:00'
---

# In the News: November 2, 2026 (Midday)


Two new benchmarks put numbers on how agents fall short when they check software. One measures how many real bugs agents find in running web apps. The other shows an agent declaring a cleanup finished while a resource it never looked for was still there.

## 1. In a vendor's benchmark, the browser tool moved agents' bug-finding results by 10% to 58%

**[QA Bench: Evaluating QA Agents on Real Web Applications](https://momentic.ai/blog/qa-bench)** · Wei-Wei Wu, CEO, Momentic · Momentic blog, November 2, 2026

Momentic built the benchmark, ran it and graded it, and its own testing agent, Mo, comes out on top. The suite has 20 product areas in four open-source apps (Gitea, Metabase, Twenty and Saleor) with 1,396 labeled bugs. Each agent got one area in a fresh sandbox for 30 or 60 minutes, and the figures are averages of three runs per area (one for Devin). At 60 minutes Mo matched 269.3 labeled bugs on average. Codex with GPT-6.1 Sol and Playwright MCP matched 86.3, and Claude Code with Opus 5.5 and Playwright MCP matched 59.0. Precision was 84.2% for Mo and 86.0% for Codex. Cost per matched bug was $2.67 for Mo, $0.80 for Codex and $4.09 for Claude Code.

The result with the most reach for people running their own agents is the browser tool. For Codex and for Claude Code with Opus 5.5, Playwright MCP found 10% to 58% more reference bugs than agent-browser, a command-line client. Which one was cheaper per bug depended on the model. The post's other finding is how little any agent found: the best recall was 19.3%, so a single pass left about four in five labeled bugs unfound.

The post lists its own limits. The labels were drafted from bug bashes by Mo, Claude Code and Codex, three of the agents being scored. GPT-6.1 Sol judged the matches. There are no error bars. Mo also had to produce a reproduction video and a written report for each bug, which the others did not, and Codex ran with web search on. We read the post and did not run the published harness.

**Why it matters:** If an agent checks your web app as part of a verification loop, the tool wiring is a variable worth testing as seriously as the model. Treat the vendor ranking as a claim and the tool-integration gap as the part to reproduce.

## 2. A new DevOps benchmark shows agents declaring a cleanup done after an incomplete survey

**[How Well Do Agents Perform on DevOps Tasks?](https://cloudopsbench.ai/blog/failure-taxonomy)** · CloudOpsBench (authors not named on the page) · CloudOpsBench blog, November 2, 2026

CloudOpsBench v1.0 has 45 AWS tasks, including decommissioning projects, securing access, migrating services and repairing failures. The post says Claude Fable 5.1 completed 29 of the 45 and GPT-6 Astra completed 26. Across the failed tasks it classes about 80% as reasoning gaps and about 20% as wrong actions, and says no task failed because an agent misunderstood a service.

Its worked example is a task to delete every queue belonging to a project. In the Fable 5.1 run shown, the agent listed SQS queues across 12 regions, deleted two, re-checked, and reported none remained. The post says no attempt by any model listed MediaConvert queues, and the independent verifier failed the run because a project queue was still there. In a bucket-migration task, the post says agents aborted an in-progress multipart upload as leftover data, which destroyed it unrecoverably.

We read the blog post and the leaderboard page. We did not open the task set or the underlying data, and the post does not name its authors.

**Why it matters:** In the example, the agent's re-check searched the same places as its first pass, so it could not find what the first pass missed. A check that lists every resource type in scope, run separately from the agent's own search, would have caught this one.