Anthropic's 90% AI-written code is a verification story
The Claude Code team ships 90% agent-written code because its checks are deterministic first and its reviewers are graded against old bugs, not because the model got smarter.
Anthropic published a number this week that reads like a benchmark score and is actually a claim about process: roughly 90% of the code produced by the team that builds Claude Code is written and reviewed by Claude. The mechanics are in an engineering post by Anthropic's Addy Osmani, and the details — reported by Amanda Caswell at The New Stack — matter more than the percentage.
The headline invites the wrong conclusion. Teams that copy it usually copy the wrong half: they add an agent and keep the review process they had before. Anthropic describes the opposite order. The agent writes, the deterministic suite judges, and a model reviewer is spent only on what tests cannot answer.
The loop is run-time, not diff-time
Claude's first move is not asking another model whether the change looks right. It runs the change — driving a browser for web work, MCP-connected simulators for mobile builds, local services for backend changes. Those instructions live in repository-specific verification skills under .claude/skills/, so any agent working in that repo inherits the same definition of "it works" before it commits.
Inside a verification loop, tests stop being regression insurance and become the agent's feedback channel — the only thing between a plausible-looking patch and a merged one. Osmani also points to property-based testing for edge cases and differential testing for behaviour changes: exactly the bugs a human waves through when the diff looks fine.
Cheap checks first, models second
The obvious objection, raised on Hacker News, is cost — if one agent writes and another reviews, you pay inference twice on every line. Anthropic's answer to The New Stack is a routing rule, not a discount:
"Deterministic checks like the build, linters, unit tests and property tests go first, because once Claude writes them they run as ordinary software and cost very little to rerun."
Model review is reserved for whether a change takes the right approach, or misses a case nobody wrote a test for. Anthropic declined to put a number on the added cost, which scales with how many of your checks are already ordinary software. A thin suite pays a model to rediscover what a linter would have caught for free.
The rule that actually matters
The harder engineering is stopping the agent from passing its own exam. Hacker News commenter xnorswap worried less about per-run cost than churn: an agent that rewrites the assertion, adds a sleep, or bolts on a special case reaches green without fixing anything. Anthropic's rule is blunt:
"If a change can modify the check that judges it, the check is broken."
So code review reads REVIEW.md from the base branch, never from the pull request under review, and a result counts only for the exact commit examined. Claude is barred from skipping flaky tests, inserting sleeps, or re-running until green; when it cannot establish a root cause, it deletes the test and documents why rather than leave a false signal. Every commit records how it was verified — including the cases where nothing verified it. A log of successes only is a marketing document; a log with the gaps is an audit trail.
Few checks, chosen per change
The team has accumulated 39 reliability checks and typically runs three to ten per change. Selection is the skill: a session-storage change triggers a check that older transcripts still load with --resume.
Commit gates require /code-review or /verify, monitored before they are enforced. Between June and September, merged pull requests to the Claude Code codebase more than doubled while the share of changes reverted fell by more than 40%. Anthropic did not attribute it to verification loops, or split production defects from regressions — correlation, not proof.
Grading the graders
The part worth stealing is how Anthropic evaluates its reviewer: replay historical incidents and reverts, rebuild the repository as it stood at review time, write a one-sentence answer key per case, and award credit only when a finding describes the actual bug. Stable merges control for false positives, and a held-out evaluation set stays off-limits for tuning. A reviewer that catches a real bug still scores a miss if the finding never reaches an engineer. Error rates are unpublished — the architecture is public, the performance is not.
What it means for a services team
For a services team, the number is not the lesson; the accountability model is. Anthropic's line — "a person is accountable for every change that ships, whoever typed it" — is a design constraint, not a disclaimer. Engineers review evidence: test output, screenshots, the verification log; after three failed rounds the agent hands over. Less typing, more deciding what counts as evidence, and more work on the suite — which also raises the stakes on what an agent may touch, as we argued in our piece on agent attack surfaces.
The checklist
Before pointing an agent at a repository:
- Can a change modify the thing that judges it? If tests, config or
REVIEW.mdsit in the same diff, the check is already broken. - Do deterministic checks run before any model call? Without build, lint, unit and property tests, model review is expensive theatre.
- Does every commit record how it was verified — including "not verified"? No log, no loop.
- Are flaky tests skippable? Ban skips, sleeps and retry-until-green, or your agent will find them.
- Is the suite incident-derived? Three checks chosen for this change beat forty generic ones.
- Do you grade the reviewer? Replay old bugs with an answer key and a control set, and check that findings reach a human.
- Is there a stop rule? Three failed rounds, hand it over.
- Who signs off? Name the person, per change.
The 90% figure will be quoted for months. The transferable part is quieter: a test suite the agent reads, a reviewer that cannot edit its own exam, and a human who owns the result. Start with the third one.