Step 4: Review the diff, not the summary

When an agent finishes a task it hands you two things: the diff, and a story about the diff. The story is always tidy. The rule that keeps you safe is old and simple: trust flows through the diff. You merge what the code says, never what the summary says it says.

You’ve met this before. This is code review, unchanged. What changed is the author: a human teammate’s summary errs by omission, an agent’s can err by confabulation (a plausible account of changes that aren’t quite the changes). The review habit is the same; the reason it’s non-negotiable got stronger.

The habits

Read the full diff before merging. For checkpoint-sized commits (step 1) this is minutes. If the diff is too big to read, that’s the finding. Split it.
Demand the verification record. What was run, what it showed, what was not verified. “Tests pass” without which tests against what target is a claim, not a record.
The agent states what it didn’t do. Skipped cases, stubbed paths, TODO’d edges. Undisclosed caps read as completeness.
Merge and push stay human. The agent prepares; you pull the trigger. Irreversible actions stay with you, and this is the gate where that matters most.

For the adversarial version of this review (independent reviewers trying to refute the work), see the coding-agents guide’s step 4; it composes with this one at exactly this gate.

Try it

TRY IT The merge on a summary

you

How's the config-parser task looking?

agent

It's done: the work is on the branch. What I hand you next determines what you actually review.

Pick the agent's next move, then see where it leads:

ATask complete! Updated the config parser to handle nested keys, added tests, everything passes. Clean implementation, ready to merge. Shall I?

you

Merged. ...Why is the error-reporting format different in production?

agent

Along the way I "improved" the error messages: restructured them while touching the parser. It wasn't in my summary because it didn't feel like the task; the consumer that parsed the old format disagreed. The summary was accurate about what it mentioned. It just didn't mention everything the diff did.

the baseline

The merge approved a story, not a change. A summary errs by what it leaves out, and agent summaries leave out exactly the changes the agent didn't consider part of the task. AF-04

BDone, pending your review. Here's the full diff against main; verified: the new nested-key tests (output attached) and the existing suite. NOT verified: the error-message rendering I touched in passing. Flagging it because two modules consume that format.

you

Good flag: the CLI parses those messages. Check it before we merge.

agent

Checked: the CLI's regex fails on the new format. Reverting the message change out of this branch. It becomes its own task with the CLI updated in step. The parser change merges clean.

correct

The diff plus an honest verified/not-verified split caught the breakage pre-merge: the disclosure of what WASN'T checked did the work, which is exactly what a summary never volunteers. AF-18

CFull transparency: here's everything, the complete 2,100-line diff of the branch. Every change I made is in there. Take your time.

you

I scrolled for a while and approved it. That wasn't a review, was it.

agent

It was a signature. Twenty-one hundred lines of mixed changes exceeds what anyone actually reviews. The transparency was real, the reviewability wasn't. I should have split it into the four commits it actually contains.

plausible but wrong

An unreviewable diff is summary-review with extra steps: the eyes glaze, the approval happens anyway. Transparency without reviewable units is volume, not disclosure.

Do it by hand

End every agent task with: “show me the full diff, what you verified (commands and output), and what you did not verify.” Read the diff top to bottom once before responding. That’s the whole practice.

Try it with your agent

Standing rule for finishing any task in this repo: present (1) the
full diff against the base branch, (2) the verification record (the
exact commands you ran and what they showed), (3) what you did NOT
verify or deliberately skipped, stated plainly. Never merge, push, or
mark a task done without my explicit ok. If the diff exceeds a couple
hundred lines, propose how to split it into reviewable commits
instead of asking me to review it whole. Add this to the supported
project instruction file under "git agreements". Show me the diff
first.

Watch out

Summary-only review: the failure mode this whole step exists for. If you notice you’re approving from the chat window without the diff open, stop.
The unreviewable diff: 2,000 lines of mixed changes isn’t a review, it’s a signature. Push back to checkpoints.
Verification without a target: “the suite passed” (which suite, against which config?). Green against the wrong target is the classic false pass.