A model built to check facts beat general reviewers at catching them

Tran tested five general-purpose models as reviewers of AI-written meeting summaries, hand-checking 35 claims against the source transcripts (23 supported, 12 not). Bigger reviewers caught more unsupported claims, up to 10 of 12, but the largest, qwen3:30b-a3b, also deleted 10 of the 23 true claims while doing it. A model trained specifically for fact-checking, bespoke-minicheck:7b, caught the same 10 of 12 unsupported claims as the best general reviewer, removed none of the true ones, and finished in about 10 seconds against a minute or more for the larger models. Only 2 of the 12 unsupported claims were caught by every reviewer tested, and one was caught by none. "I would not let a general reviewer silently delete claims on its own," Tran writes, after testing flagging and multi-reviewer voting as alternatives and finding each trades missed errors for lost content in a different way.

Why it matters: A general-purpose model asked to double-check another model's output is not automatically a safety layer. This is a case for choosing a reviewer trained for the specific judgment a verification step needs, rather than assuming a larger model is a better check.