Why an agent's answer can look right and still be wrong

A healthcare analytics agent was asked for Medicare Advantage readmission rates tied to a specific diagnosis and date range. It returned a plausible number that was wrong three separate ways: it mixed in the wrong plan type, used an outdated diagnosis code grouping, and misread the date range. "Our end-to-end evaluation missed all three, because the query ran and the output looked plausible," Srinivasan and Xiong write. Their proposed fix is a reliability ladder: six governance layers, terminology, routing, agent intent, semantic context, execution, and result, each owned by a different team and each backed by what they call a truth contract stating what must hold true, how it's measured, and what happens when it fails. A failure taxonomy assigns each kind of violation a code before it ever happens, so a broken contract routes automatically to whoever owns the fix.

Why it matters: "A final answer can be correct for the wrong reasons, or wrong even when every stage reports success," the authors write, which names the failure mode more precisely than most reliability write-ups manage. Testing only the final output, which is how most teams currently evaluate agents, will not catch this class of error.