---
title: 'In the News: November 21, 2026 (Midday)'
description: "A reproducibility study finds common ways of choosing an agent's self-written skill edits pick the better one about half the time."
canonical_url: 'https://darkfactory.dev/news/2026-11-21-midday'
markdown_url: 'https://darkfactory.dev/news/2026-11-21-midday.md'
collection: news
date_published: '2026-11-21T13:01:00-05:00'
date_modified: '2026-11-21T13:01:00-05:00'
---

# In the News: November 21, 2026 (Midday)


Two of today's items concern agents that rewrite their own prompts, skills and memory. A study of more than a thousand such runs finds the usual way of deciding which edit to keep works no better than chance, and that the largest reported gains came from benchmark shortcuts. A Meta team shows a loop that held up on a production ranker, where every candidate had to beat a noise baseline trained alongside it. LangChain's CEO describes a background agent that updates company knowledge from agent traces.

## 1. Self-improving agents pick the better skill edit about half the time

**[The Resolution Hypothesis: Agent Evolution's Key Is Identifying Improvements](https://arxiv.org/abs/2611.23068)** · Yuanhao Zeng, Kan Ren and nine co-authors, ShanghaiTech University, ByteDance, BIGAI and SUSTech · arXiv, submitted November 17, 2026

The authors ran seven agent-evolution methods (prompt, skill, memory and harness evolvers, including GEPA and Meta-Harness) on six models: 1,030 independent evolution runs and over 2.1 million test trajectories. On a pool of 100 candidate skills scored across 11 Terminal-Bench tasks, selection on execution scores preferred the better skill 49.7% of the time. Regressions were 30.5% of the pool but 62.1% of what got accepted. LLM judges reading the full trajectory did no better. Good skills were there to find: in a separate pool, the strongest shared skill scored 57.5% against a pool mean of 49.2%. In re-evaluated SkillOpt-Lite and GEPA runs on Terminal-Bench, the evolvers' own validation scores rose 6.9 to 9.0 percentage points while independent re-evaluation put the gain between +0.4 and -3.2. In the authors' words, "In-loop scores improve under this process; agent capability does not."

The largest gains reported elsewhere did not survive an audit. Across 27 model and benchmark pairs with a reported gain above 3 points, adding the single most responsible rule to the unevolved agent recovered a median 90% of the gain. That rule typically exploited an evaluation shortcut, such as a grader that credits only one answer format. Repairing those evaluations cut the median gain from 17.6 to 0.8 points. Across 171 results in nine papers, the median margin over the strongest competitor was 3.4 points, and none of those results reported a variance estimate. At that margin, Terminal-Bench needs about 23 runs per method to tell two methods apart at 80% power. The trajectory data is not yet public; the authors say it will be released on acceptance.

**Why it matters:** If an agent edits its own skills or harness and keeps whatever scored higher on one rollout, this paper's evidence says much of what you keep is noise. Score candidates on repeated rollouts, report the run-to-run spread, and check whether a gain traces to one rule that fits the grader. In a preliminary analysis, the authors found checks on intermediate behavior carried 13 to 57 times less evaluation noise than pass/fail verdicts.

## 2. Meta's ML engineering agents kept a change only if it beat a noise probe

**[MLE Agents: Autonomous ML Engineering in Large-scale Recommendation Systems](https://arxiv.org/abs/2611.23910)** · Matthew Li, Albert Jiongqian Liang and Dilip Raj, Meta Platforms · arXiv, submitted November 18, 2026 (NeurIPS 2026 AutoMLR Workshop)

The team built agents that propose a change to a search ranker as code, launch a 64-GPU training job, read the offline metrics and decide whether to keep it. Each iteration trained a nominal seven candidates plus a variance probe in parallel. A candidate was accepted only if it beat its same-iteration probe by at least 0.0005 normalized entropy. Over 31 days and roughly 360,000 GPU-hours, the second-generation system completed 1,097 candidate runs and kept 325. The resulting model raised search session volume by 0.23% (±0.15) in an A/B test of about a week. It also regressed a time-to-render latency guardrail; the authors note that "an agent optimizing a pure quality objective does not surface such trade-offs on its own."

The controller model, Claude Opus 4.8, was held fixed, so the authors attribute every reported improvement to harness and objective design. That design was a calibrated objective, parallel runs, and context rules: a "What Didn't Work" blocklist, one change per candidate, strict output formats, and feeding back only recent winners. Malformed edits fell from 33.8% to 0.8%, and the acceptance rate rose from 15.6% to 29.6%. The objective and search budget also changed between versions, so the authors say this is not a controlled ablation. Engineers still set the objective, steered the search (in the first version, after a 0-for-8 streak on optimizer tuning) and reviewed the final model.

**Why it matters:** This is the kind of gate the first item finds missing from common agent-evolution methods: a noise measurement trained in the same batch as every candidate, and an acceptance bar set from it. It is also a first-party report of an agent loop's changes reaching a live A/B test, with the latency cost stated next to the gain.

## 3. LangChain describes a "knowledge factory" for agents

**[How to build a knowledge factory](https://x.com/hwchase17/status/2124181216719151585)** · Harrison Chase, CEO, LangChain · X Article, November 21, 2026

Chase separates context, what an agent has during a task, from knowledge, "the reusable understanding we maintain" so the task is done correctly. He presents the idea as the knowledge-work counterpart of the software factory and splits knowledge into skills, domain knowledge and guidance on where data lives. The factory has three parts: a tool agents use to flag missing or wrong knowledge, storage in whatever format suits, and an agent that maintains the knowledge. On that last part he writes that "you will want a curator agent that runs in the background, on some cadence, to update this knowledge." It reads raw sources, agent traces and the flags. He argues that leaving upkeep to the agents doing the work will make the knowledge "unruly and full of contradictions," so they should flag problems and leave the fixing to that agent. Chase says LangChain's internal GTM and content studio agents and its LangSmith Engine work this way. The post gives no measurements and ends with an offer of LangChain's help.

**Why it matters:** The split between agents that report gaps and one agent that edits is a design choice you can copy without the product. The post does not say how a background update is checked before other agents rely on it. Item 1 suggests that check is where loops of this kind tend to fail.
