xs.dev

Adversarial verify

Before I trust a piece of work, reviewers try to break it. Every finding is then checked against the facts, because reviewers are wrong too.

Written
18 November 2026
Reading
2 min read
Share
Adversarial verifyReviewers read a draft and try to break it. Their findings go to a check against the facts, which is the gate. A finding that is real is fixed in the draft. A finding that is not real is rejected with one line that says why. A dashed arrow leads from the fix back to the draft: the next revision.readfindingsrealnot realnext revisionDRAFTspec or codeREVIEWERStry to break itCHECKagainst factsFIXin the draftREJECTEDone line whyAdversarial verifyReviewers read a draft and try to break it. Their findings go to a check against the facts, which is the gate. A finding that is real is fixed in the draft. A finding that is not real is rejected with one line that says why. A dashed arrow leads from the fix back to the draft: the next revision.readfindingsrealnot realnext revisionDRAFTspec or codeREVIEWERStry to break itCHECKagainst factsFIXin the draftREJECTEDone line why
Reviewers attack the draft. Each finding is checked against the facts: a real one is fixed, the rest get one line that says why not.

Use it When a mistake would be expensive, for example anything that guards private data.

The problem

The agent that wrote something is the worst one to check it. It reads its own work and sees what it meant. Asking it “are you sure?” gets a yes.

How it works

  1. A draft exists: a spec, or code.
  2. Reviewers get one job, which is to find what is wrong with it. They are not asked whether it is good.
  3. Every finding is checked against the facts: the documentation, the repo, the real system. A finding is a claim, not a truth.
  4. A finding that holds is fixed in the draft. A finding that does not hold is rejected, with one line that says why.
  5. The result is the next revision, and the list of what was accepted and rejected stays with it.

The same idea works as a test that runs every time. The workbench has a canary test: it writes a private note that holds a random word, then reads every public page and every public endpoint and fails if the word shows up anywhere.

What it costs

A full extra pass over the work, and then the checking, which is the slow part. Each finding has to be looked up, one by one.

Where it fails

Without step 3. Reviewers that are told to find problems will find problems, including ones that are not there. If every finding is accepted, the draft gets worse. On the workbench spec some suggestions were turned down after that check, each with its reason.

It also only covers what the reviewers thought to attack. A clean review is not a proof.

Where I used it

On the spec of the AI lab, before any code was written. One finding that held: the test login for agents had the same rights as me on a preview that would hold my real notes. It now sees only the test rows it made itself. One that did not hold: a request to build the packages in another order.

Used inAI lab

Next: Loop until dryAll patterns

Contents

  1. 01The problem
  2. 02How it works
  3. 03What it costs
  4. 04Where it fails
  5. 05Where I used it

Contact

    ↑↓ to moveEnter to open/ opens this anywhereEsc for the resultsEsc twice to close