---
title: 'In the News: November 3, 2026 (Extra 2)'
description: 'A Stanford re-test finds 44% of 50 cited LLM findings expired on current models, while judges favoring their own answers did not fade.'
canonical_url: 'https://darkfactory.dev/news/2026-11-03-extra-2'
markdown_url: 'https://darkfactory.dev/news/2026-11-03-extra-2.md'
collection: news
date_published: '2026-11-03T16:15:00-05:00'
date_modified: '2026-11-03T16:15:00-05:00'
---

# In the News: November 3, 2026 (Extra 2)


A Stanford team re-ran 50 widely cited findings about LLM behavior on 20 models and found that 44% no longer hold. Failures that models can be scored against faded fastest. Tendencies that are harder to score, including a judge preferring its own answers, mostly stayed.

## 1. A Stanford re-test finds 44% of 50 cited LLM findings have expired, while self-preferring judges persist

**[The shelf life of LLM findings](https://zou-group.github.io/llm-claims-shelf-life/)** · Owen Queen and James Zou, Stanford University · Project page, November 2, 2026

The authors repeated 50 published findings on 20 models released between July 2024 and September 2026 from eight developers, then fitted how each effect changed with release date. By September 2026, 44% of the claims had expired, which the authors define as an effect at one fifth or less of the original. Another 34% had weakened, 16% persist and 6% strengthened. Expired findings include facts lost in the middle of long contexts and models abandoning correct answers when challenged. Basic prompt injection weakened to about one quarter of its original effect. Judges favoring their own answers persisted, at 1.09 times the paper's effect for Claude Sonnet 5.5 and 2.88 times for GPT-6.1 Sol.

The authors offer one explanation: "these failures have a right answer that developers can measure and train against." They also say high-profile failures "might have been specifically patched." They call the work preliminary. It covers 50 claims, the confidence intervals are wide, and each model ran at its provider's default reasoning setting rather than a common budget. Results also differ by model: Qwen3.7-Flash fabricated 69% of references in the citation test, against about 1% for Sonnet 5.5. This summary draws on the page text, not the individual claim cards or the underlying data.

**Why it matters:** If a harness has one model grade another model's work, the self-preference effect is one the study found has not faded. The study does not test fixes, but a judge from a different model, or a check that does not depend on a model, avoids relying on it. Workarounds built for older failures, such as moving key facts out of the middle of a long prompt, are worth re-testing against the model you run now.

---

## Thread watch

_Discussions gathering force. No primary read yet, so these are reported as discussions, not as findings._

- **[Revolut moves to self-hosted GLM](https://news.ycombinator.com/item?id=50350004)** · Hacker News · 177 points and 153 comments at about 16:00 EST, posted at 09:53 EST. The thread links an X post summarizing a conference talk by a speaker it names as leading AI operations at Revolut. The talk itself was not read for this line, so it reports a discussion, not a finding, and the cost and productivity figures in the post are not repeated here.