Most agent sessions end the same way. The agent produces a summary of what it did. Files changed, tests passing, a short account of the approach. You skim it, it looks reasonable, you move on.
That summary is the least informative artifact in the whole run. It is a restatement of the diff in prose, written by the party with the least incentive to find fault in it, at the moment it is most convinced the work is finished.
The replacement I run is adapted, not invented. In early July a r/ClaudeAI thread titled "I end every AI session with two questions" made the rounds, and the commenters improved it as they went: one contributed a version of the production-fragility question, another the rule that every claim in an answer has to name its own check. I took that shape and scripted it into my harness, asked every time, in the same words, not typed by me when I remember. The change in what I catch has been larger than any rule file I have written.
The summary is a description, not an evaluation
The problem with the closing summary is structural. You are asking a system that just optimized for task completion to now report on task completion. It will report success, because it stopped when its own success criteria were met. If those criteria were wrong, the summary is confidently wrong in exactly the same way, and reads identically to a correct one.
There is no signal in it. A good run and a bad run produce the same document.
The failures I actually care about do not look like errors. They look like decisions. A retry loop that silently swallows a specific exception. An assumption that a list is non-empty. A cache key missing a tenant identifier. In every case the code runs, the tests pass, and the summary says the feature was implemented. Nothing about the closing artifact is capable of surfacing any of it.
The two questions
What is the part of this you are least confident about, and why?
This works because it changes what the model is optimizing for. Asking "is this correct?" invites a defense, and you get one. Asking which part is weakest presupposes that a weakest part exists and asks only for its identification. It is a much easier request to satisfy honestly, and the answers are specific in a way that general self-review never is.
The answers I get are things like: I assumed the upstream response always includes this field, and I did not verify that. Or: I matched the existing pattern in this file, but that pattern looked wrong to me and I followed it anyway. That second kind is worth the whole protocol on its own.
If this breaks in production, what breaks first, and what would you look at?
This one is doing something different. It forces a shift from the construction frame to the operational frame, and the two frames surface different problems. While building, the question is whether the code does the thing. While operating, the question is what happens under conditions the builder did not choose: concurrency, partial failure, bad input, the third retry.
It also produces something immediately useful. If the answer is a specific log line or a specific table, I know my observability is adequate. If the answer is vague, I have learned that this change shipped without a way to diagnose it, which is worth knowing before rather than after.
Make the answers checkable
An interrogation that produces prose is an improvement over a summary that produces prose, but not a large one. The version that earns its cost adds a constraint: every claim in the answer must be either verifiable or explicitly marked as unverified.
So "I assumed the field is always present" is not an acceptable stopping point. The follow-up is: is that assumption checked anywhere in this change, and if not, is it checkable now? Sometimes the answer is a one-line guard. Sometimes it is a test. Sometimes it cannot be known without production data, and then it gets marked as a known unverified assumption and written down somewhere durable.
The list of known unverified assumptions has turned out to be one of the more valuable artifacts my factory produces. It is the closest thing I have to a map of where the system is likely to fail.
Why this beats writing another rule
The reflex when an agent does something wrong is to add a rule. Do not swallow exceptions. Always include the tenant in the cache key. The rule file grows, and each rule is individually reasonable.
The problem is that rules are prospective and specific. They prevent the failure you already had. The interrogation is retrospective and general, and it surfaces failures you have not yet imagined, which is the only category that matters after the first few months.
There is also a cost asymmetry worth being explicit about. Every rule you add is in context for every future session, forever, whether or not it is relevant. The interrogation costs a fixed amount at the end of a session and nothing at any other time. Rules scale badly. A closing protocol does not.
I still write rules. I write fewer of them, and mostly for things the interrogation caught twice.
Where this argument is weak
The largest objection is the honest one: this is the builder evaluating its own work, which is exactly the thing I do not think you should rely on. A model that misunderstood the problem will report low-confidence areas within its misunderstanding. It cannot flag an assumption it does not know it made. The interrogation catches known unknowns and is structurally blind to the other kind.
So I want to be careful about what I am claiming. This is cheap triage, not certification. It costs a few hundred tokens and surfaces candidates for real verification. It is not evidence that anything works, and a factory that treated it as evidence would be worse off than one with no protocol at all, because it would feel checked. Independent verification is a separate problem and this does not touch it.
Second, the answers degrade if the questions become predictable within a single long session. Asked repeatedly in the same context, the responses converge toward a template. I ask once, at the end, and I have not found a way around that limitation.
Third, I cannot give you a measurement. I know it catches things because I have the list of things it caught. I do not know the base rate, and I do not know how many of those I would have caught anyway during review. Anyone claiming a percentage here is guessing.
What this changes in my factory
The interrogation is a scripted harness step, not something I type. This is the entire difference between a protocol and an intention. When it depended on me remembering, I ran it on the sessions where I already suspected a problem, which is precisely the wrong sample.
The wording is fixed and version-controlled. When I change it, that is a deliberate change with a reason, not drift.
Answers are captured to a file, not left in the transcript. The transcript is going to be archived and rarely reread. The unverified-assumptions list gets read constantly.
Any claim in an answer that could be checked gets checked in the same session, while the context is still loaded. Deferring this means it never happens.
And a low-confidence answer that names a specific line is treated as a review request, not as a note. That is the one that has paid for the whole thing.
The script
The version I run, close to verbatim:
Before you finish: name the single part of this change you are least confident about, and say why. Be specific about the file and the line. Then: if this breaks in production, what breaks first, and exactly what would you look at to diagnose it? For each claim you make, say whether it is verified by something in this change, verifiable now, or unverified. Do not summarize what you did.
That last sentence matters more than it looks. Without it you get a summary with two questions appended, which is a summary.