Sign Off on Consequences, Not Lines
Replace line-by-line review with an accountability statement backed by an evaluator.
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 6
- Confidence
- 70%
Code review rests on a promise almost nobody keeps anymore: that a human read every line. With agents producing large pull requests, the honest standard shifts from reading to accountability. A reviewer signs a specific statement: I understand the consequences of this change and I will stand behind it in production. If you cannot say that from reading, you earn the right to say it by building the evaluator instead, the test harness, the simulations, the proofs and the type checkers that produce confidence without line-by-line comprehension. The human function moves from author to verifier, mirroring the role verification data plays in training the models themselves. Pair the sign-off with the thousand-day question: not can this ship today, but will it still be secure, tested, performant and worth maintaining a thousand days from now, and are you still willing to spend tokens keeping it there.
Origin
Extracted from Naval, where Guillermo Rauch describes what he wants from infrastructure engineers reviewing pull requests full of AI-generated code.
Core principles
- 01Nobody reads every line of a large agent-generated pull request; pretending otherwise is theatre.
- 02Understanding consequences is the real bar, not comprehension of syntax.
- 03If you cannot read your way to confidence, build your way to it.
- 04Someone gets paged when it breaks; that person is the signer.
- 05The human role in the stack is shifting from author to verifier.
How to run it
- 1
Retire the line-by-line fiction
State openly that reading every line is no longer the review standard, because agent-authored changes routinely exceed what any reviewer will genuinely read.
Pro tip Say it in writing so reviewers stop performing thoroughness they are not delivering.
Watch out This is a change of standard, not a lowering of it. Do not announce step one without steps two and three.
- 2
Define the sign-off sentence
Write the exact statement a reviewer is asserting: I am signing off on understanding the consequences of this change in production, and I will support it if it goes wrong.
Pro tip Put the sentence in the pull request template so it is affirmed, not assumed.
- 3
Build the evaluator when reading fails
Where the change is too large or too subtle to understand by reading, the reviewer's job becomes constructing the test harness, simulation, proof or type-level constraint that supplies the confidence instead.
Pro tip Spend agent tokens on the evaluator, not just the feature. It is the durable asset.
Watch out An evaluator that passes by construction measures nothing. Make it capable of failing.
- 4
Name who gets paged
Every merged change has a human who will be woken up if it fails. That person is the one entitled to sign, which keeps accountability attached to a name rather than a process.
- 5
Ask the thousand-day question
Before merging anything load bearing, ask what this code looks like a thousand days from now: secure, tested, production grade, performant, and still worth spending tokens to maintain.
Pro tip Zero to one is now easy; the honest cost estimate lives in years two and three.
Watch out Most AI-era software estimates ignore maintenance entirely because generation felt free.
- 6
Reframe the role as verification
Tell the team explicitly that the job is moving from producing the artifact to verifying the stack, the same way good verification data is what trains the models.
In the wild
An agent produces a large change to a production infrastructure service. The reviewer cannot honestly claim to have read all of it. Instead they write the simulation and test harness that exercises the failure modes they care about, run it, and then sign the statement that they understand the consequences of shipping it and will support it in production.
→ The change merges with real, stated accountability rather than a rubber-stamped approval.
A service was generated quickly and works. Before it is adopted as a dependency, the team runs the thousand-day question against it: is it secure, is it tested, is it performant, and is anyone willing to keep investing tokens in maintaining it. Two of the four answers are no, so the service is either hardened or kept out of the critical path.
→ Fast-generated code is triaged before it becomes a permanent liability.
Common mistakes
Approving without an evaluator
Dropping line-by-line reading without adding tests, simulations or type-level guarantees just removes the only check that existed.
Confusing reading with understanding
Reading every line of a large change rarely produces an accurate model of its production consequences, which is the thing the sign-off is actually about.
Costing only the generation
Zero to one is cheap now, so teams underprice the years of security, testing and maintenance that follow and end up unable to staff them.
Is it for you?
Best for
Infrastructure and platform teams shipping large volumes of AI-assisted code into production systems that page someone at 3am.
Not ideal for
Small throwaway prototypes and personal tools where nothing runs in production and no one is on call.
From the transcript
“I want engineers to be able to say I understand doesn't necessarily mean that you've read every line of the of the PR.”
“I wrote the test harness the simulations the proofs the type checkers etc to be able to say even without reading this I have confidence…”
“creating software is really easy 0ero to one but think about a thousand days from now what does what does your software look like is…”
From the episode
Full Episode: The AI Industrial Revolution