Verification: The One That Actually Matters

Most prompt advice is cope. People spend an hour polishing a paragraph, adding “you are a senior engineer” and “think step by step,” tweaking word order like it’s a spell. Then they hand the model a giant task, walk away, and come back to something that looks done and isn’t.

There’s a better use of that hour. It comes from a Boris Cherny interview with the person who built Claude Code, and it’s the single habit that moved my success rate more than any prompt trick I’ve tried.

The thesis: give the model a mirror

Here’s the claim, straight from the interview: the number one predictor of whether a model finishes a long task correctly isn’t how good the prompt is. It’s whether the model has a way to verify its own work.

Read that again, because it inverts how most people spend their effort. A plain, almost lazy prompt paired with a real verification loop beats a beautifully engineered prompt with no way to check the result. Every time.

The mental model I keep is: build the model a mirror. On its own, the model is writing with its eyes closed. It generates code, believes the code is good, moves on. It has no ground truth, so its confidence and its correctness drift apart until you get that classic “here’s your fully working feature” attached to something that doesn’t compile.

A verifier is the mirror. It’s an objective signal the model can look into after every move and see the difference between what it thinks it did and what actually happened. Tests, types, a screenshot diff, a linter — anything that answers “did that work?” without a human in the loop.

Stop optimizing prose. Start building checkers.

Why prose has a ceiling and checkers don’t

Prompt tuning has diminishing returns fast. You can nudge tone and structure, but you’re steering a probability distribution — you never guarantee anything. There’s no assertion in an English sentence. “Make sure the tests pass” is a wish. npm test exiting non-zero is a fact.

The difference matters most on long tasks, because errors compound. Step 3 builds on a wrong step 2, step 8 builds on all of it, and by the end you’re deep in a hole that a single early check would’ve caught. Without verification, the model can’t tell it’s in the hole. With verification, it hits a wall, reads the failure, and course-corrects on its own — no message from you required.

That’s the whole game: turn “trust me” into “here’s the exit code.” Your job shifts from author to environment designer. You’re not writing better instructions; you’re building a room where wrong answers can’t hide.

The kinds of verifier

Any objective, automatable check qualifies. Four carry most of the weight.

1. A test suite — the red/green loop

The cleanest mirror there is. Tests either pass or they don’t, and the failure output tells the model exactly what broke. The move is to make the tests the target, not a footnote.

Implement retryWithBackoff in src/retry.ts so the suite in
src/retry.test.ts goes fully green.

Run `npm test -- retry` after every change. Do not tell me it's
done until that command exits 0. If a test fails, read the actual
assertion output, fix the code, and re-run. Repeat until green.

Notice what that prompt is not doing. It’s not describing backoff algorithms or explaining exponential jitter. The tests already encode the spec. The prompt just points the model at the mirror and tells it to keep looking until the reflection is green. If you have the spec but not the tests yet, flip it — have the model write the tests first, get you to eyeball them, then implement against them.

2. Typecheck and lint — the free verifier you already own

If you’re in a typed language, you have a verifier sitting right there doing nothing. tsc --noEmit, cargo check, go vet, mypy, ruff. These catch a whole class of “looks right” bugs — wrong shapes, missing cases, null holes — for basically zero setup.

Refactor the User type to make `email` optional and thread the
change through every consumer.

After each file you touch, run `npx tsc --noEmit`. Treat any type
error as a failure to fix before moving on. Do not add `any` or
`@ts-ignore` to make errors disappear — fix the actual type.

That last line matters. A verifier is only as honest as the escape hatches you leave open, and models will absolutely reach for any or a skipped test to make the red go away. Close the hatch in the prompt, or better, in a hook (below).

3. Visual / screenshot diff — the mirror for UI

Tests and types don’t know if your button is centered or your layout exploded. For UI you need a check that looks at pixels. This is where an MCP setup earns its keep: give Claude Code a browser-automation MCP server (Playwright-style), and the model can navigate, screenshot, and compare against a reference.

The login page should match design/login-reference.png.

Loop: start the dev server, use the browser MCP to open /login,
take a screenshot, and compare it to the reference. List every
visible difference — spacing, color, alignment, font. Fix the CSS,
reload, screenshot again. Keep going until the screenshot matches
the reference. Show me both images before you call it done.

The model now has eyes. It’s not guessing whether justify-content did what it hoped — it’s looking at the result and diffing against truth. Slower than a unit test, but for “does this actually look right,” nothing else substitutes.

4. Property tests — checking the rule, not the example

Example tests check specific inputs. Property tests check invariants — “for any input, this must hold” — and let a framework hunt for the counterexample. They’re the strongest mirror for anything with a contract: parsers, serializers, sort orders, money math.

Add fast-check property tests for parseAmount:
  - for any valid amount string, parse then format returns the
    original (round-trip)
  - parse never returns NaN for inputs matching the money regex
  - negative amounts stay negative through the round-trip

Run them. When fast-check finds a shrinking counterexample, that's
a real bug in parseAmount — fix the function, not the test.

“Fix the function, not the test” is the load-bearing sentence in half these prompts. The instinct under pressure is to soften the check. Don’t let the mirror lie.

Wiring it in so you don’t have to nag

Putting “run the tests” in every prompt works, but you’ll forget, and the model will sometimes “forget” too. The durable fix is to make verification part of the environment itself, using Claude Code’s hooks — shell commands the harness runs automatically on events. They live in .claude/settings.json.

Here’s a real one. PostToolUse fires after a tool runs; this config watches for edits and runs typecheck + tests right after, feeding any failure straight back into the model’s context.

{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Edit|Write|MultiEdit",
        "hooks": [
          {
            "type": "command",
            "command": "npx tsc --noEmit 2>&1 | head -40",
            "timeout": 120
          },
          {
            "type": "command",
            "command": "npm test --silent 2>&1 | tail -40",
            "timeout": 180
          }
        ]
      }
    ]
  }
}

Every time the model edits a file, the harness runs tsc and the test suite and pipes the output back. The model didn’t have to remember to check — the room checks for it. Red output shows up in context, and the model reacts to it like any other tool result. This is the mirror bolted to the wall so it can’t be skipped.

Two practical notes. Keep the output trimmed (head/tail) so a wall of logs doesn’t blow your context. And mind the timeout — a slow full suite on every edit gets annoying, so scope it to the fast tests, or the affected package, and save the full run for the end.

That “end” is the other hook worth having. The Stop hook fires when the model tries to wrap up the turn. If its command exits non-zero, the stop is blocked and the model is told to keep working — a hard gate on declaring victory.

{
  "hooks": {
    "Stop": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "npm test --silent > /dev/null 2>&1 || { echo 'Tests are still failing. Do not stop — read the failures, fix them, and continue.' >&2; exit 2; }",
            "timeout": 300
          }
        ]
      }
    ]
  }
}

Exit code 2 with a message on stderr is the signal the harness reads as “block and tell the model this.” So the model literally cannot end the turn with a red suite. “Done” now means verified done, enforced by the environment instead of your goodwill. It’s the closest thing to a bouncer for the phrase “should be working now.”

You can pair this with a permissions block to keep the model on-rails while it loops — allow the test and typecheck commands outright so it isn’t stopping to ask, deny the destructive stuff:

{
  "permissions": {
    "allow": ["Bash(npm test:*)", "Bash(npx tsc:*)", "Bash(npm run lint:*)"],
    "deny": ["Bash(rm -rf:*)", "Bash(git push:*)"]
  }
}

Now the verify loop runs friction-free and the model can’t wander somewhere dangerous while it’s heads-down.

Making the loop the default: a slash command

Hooks enforce verification. A custom slash command makes the whole loop — do, verify, fix, repeat — the thing you invoke on purpose. Claude Code reads command files from .claude/commands/<name>.md; the filename becomes /name, and $ARGUMENTS is whatever you type after it.

Here’s the full .claude/commands/verify-then-continue.md I keep in projects:

---
allowed-tools: Bash, Read, Edit, Write, Grep, Glob
description: Do the task, then loop do->verify->fix until an objective check passes.
---

Task: $ARGUMENTS

Work this as a verification loop, not a single pass:

1. IDENTIFY THE VERIFIER first. Before writing code, state the
   objective check that proves this task is done — a test command,
   a typecheck, a lint, a screenshot diff, or a script that exits
   non-zero on failure. If no verifier exists, write one (a failing
   test or an assertion) and confirm it fails for the right reason.

2. IMPLEMENT the smallest change aimed at passing that check.

3. RUN THE VERIFIER. Paste the actual command and its real output.
   No paraphrasing, no "should pass now."

4. If it fails: read the actual error, fix the cause (never weaken
   the check, never add `any`/`ts-ignore`/`.skip`), go to step 3.

5. If it passes: run the full suite + typecheck once more to catch
   regressions. Only then report done, and show the passing output
   as proof.

Do not ask me for confirmation between iterations. Loop on your own
until the verifier is green or you hit a real blocker — if blocked,
tell me the exact failing output and what you tried.

Then it’s just /verify-then-continue add rate limiting to the /api/send endpoint. The command front-loads the discipline: name the mirror, build it if it’s missing, and don’t stop staring into it until it’s green. Step 1 is the one people skip and the one that matters most — decide what “done” objectively means before you start, or you’ll rationalize your way to done later.

The extreme example: two weeks, unsupervised

The interview has a story that sounds made up until you sit with the mechanism. Someone rewrote an Electron app into native Swift. The verifier wasn’t a test suite — it was the old app itself. The model would run both versions side by side, screenshot each, pixel-diff them, and treat any visual difference as a bug to close. Match the Electron app, pixel for pixel, screen by screen.

That loop ran roughly two weeks, largely unsupervised.

Sit with why that’s even possible. Nobody can babysit a two-week rewrite. The only reason it worked is the model had a mirror that never got tired and never lied: does the Swift screen look identical to the Electron screen? Yes or no, checkable a thousand times without a human. The reference app was the spec, the screenshot diff was the assertion, and the model just closed the gap over and over.

That’s the ceiling of this idea. A good enough verifier plus a loop turns a model into something that can grind on a hard problem far longer than you’d ever supervise it. No clever prompt gets you two weeks of autonomy. A mirror does.

Test-time compute: why this is the actual lever

Here’s the framing that made it click for me. We’re used to thinking intelligence scales with model size — bigger model, smarter answers. But there’s a second axis: how many verify-and-retry cycles the model gets to spend on your problem. That’s test-time compute — thinking at the time of answering, not just what got baked in during training.

A verification loop is test-time compute you control. Each do-then-check-then-fix pass is another increment of effort spent converging on a correct answer instead of a plausible one. A model that can try, check, and retry ten times will beat a smarter model that gets one blind shot — because the loop lets it find the right answer instead of having to know it up front.

Per the interview, this is where the frontier is heading: models like a future Opus 5 are described as getting sharply better at exactly this kind of long, self-verifying autonomous work — treat that as a claim from the conversation, not settled fact. But you don’t need to wait on it. The lever is already in your hands today, and it’s not the model you picked. It’s whether you gave it a mirror and room to loop.

The failure mode this kills: “the code looks right”

The most expensive four words in this whole practice: the code looks right.

It’s the default failure, and it’s seductive because it’s usually 90% true. The model reads its own output, pattern-matches it against a thousand correct examples, and the vibe checks out. So it reports done. The vibe is not a verifier. “Looks right” is the model grading its own homework with the answer key it wrote itself.

You beat it by refusing to accept a subjective signal where an objective one exists. When the model says “this should work now,” your reflex is one question: what command proved that? If the answer is “none, but it looks correct,” it isn’t verified, full stop — changed, maybe; verified, no. Those are different states and blurring them is how you ship the bug.

Force the objective signal. Not “the function handles empty input” but “here’s the test with [] and here’s it passing.” Not “the layout is fixed” but “here’s the screenshot.” The hooks above do this mechanically — the model can say looks-right all it wants, but the Stop hook only cares about the exit code. That’s the point of moving verification into the environment: it stops being something the model can talk its way past.

The checklist

Adding a verifier to any task, before you write a line of implementation:

1. Name the mirror. What objective check proves this is done? Test, typecheck, lint, screenshot diff, or a script that exits non-zero on failure. If you can’t name one, that’s the first thing to build.
2. Make it fail first. Confirm the check is red for the right reason before you fix anything. A green check that was always green verifies nothing.
3. Make it cheap to run. The model should be able to run it after every change without a 5-minute wait. Scope to the fast path; save the full suite for the gate.
4. Point the model at it. In the prompt or a /verify-then-continue command: run it, read real output, fix the cause, repeat.
5. Close the escape hatches. No any, no ts-ignore, no .skip, no weakening the check to make red go green. Say it out loud.
6. Bolt it to the wall. For anything you’ll do more than once, move it into a PostToolUse hook and gate “done” behind a Stop hook. Stop relying on memory — yours or the model’s.
7. Demand proof, not vibes. “Done” means the passing output is on the screen. “Looks right” is not a status.

Do the flashy prompt engineering if you enjoy it. But if you only change one habit, change this one: stop writing the perfect instruction and start building the mirror. The model can’t see its own work. Give it eyes, and get out of the way.