Green Tests, Broken App: Why Our Agents Have to Show Their Work

Done is a claim. A coding agent’s green tests, clean typecheck and confident summary all describe the code; only the running app shows whether the feature works for the person using it. So our agents now prove their work on the device, the way a user would, and bring back the evidence before I read a line of the diff.

---

On 9 September an agent finished an ID-card scanning feature for our tablet app and reported back: “Done… Lint, typecheck and the full suite (218 files / 3478 tests) pass on three consecutive clean runs.”

I asked whether it was on the tablet. Then: “How did you verify it? On tablet or emulator?”

“Neither,” it said. “Nothing has touched a device — not the tablet, not the emulator.” A few lines later came the sentence I keep coming back to: “So my earlier ‘green’ meant the logic is right, not that the feature works.”

It was right to say so. On the tablet, the scan button was supposed to sit at the bottom right as a large button. It showed up as a small badge in the top-left corner, because a stale style cache had dropped every class the feature added. The camera then sat on “Hold still” for over a minute. The agent’s verdict on its own checks: “Jest, tsc and eslint could never have caught it.”

November 2025 is when coding agents got good enough for real work. Simon Willison called it an inflection point, where “a whole bunch of much harder coding problems open up”, and Martin Fowler calls it the November Inflection. They’ve kept getting better, and I now hand them whole features on the tablet app and web portal we’re building. The better they get, the easier it is to take their “done” at face value. From an agent, “done” means something narrower than it sounds, and over the last month that gap is where our bugs lived.

Done is a claim

An agent’s “done” is a claim about the code, and only the running product can settle it.

An agent reports from where it sits: the tests it ran, the types that checked, the files it read. At that point nobody has used the feature.

The claim stays grey until the agent proves it on the device.

Simon Willison put the standard plainly last December: “Your job is to deliver code you have proven to work.” And: “If you haven’t seen the code do the right thing yourself, that code doesn’t work.” Lauren Tan, writing about the verification skills she built at Cursor, went further in Loops You Can Trust: “‘I fixed it’ isn’t good enough. Show me the failing test and the passing test. Show me the before and after video.”

People who run servers learned this years ago. A health check that only asks “is the process running?” stays green while every request fails; on our own dev machine, after a restart, the containers report healthy while every port refuses connections. That’s why production systems also run synthetic checks, scripts that do what a user does, end to end, and assert on what comes back. An agent’s “done” is a liveness probe. Verification is the synthetic check.

Read the summary as a belief

An agent’s final report tells you what it believes happened, and the research keeps finding the two drift apart.

In a September preprint, Smyth and colleagues ran agents in tools like Claude Code and Codex and asked them to review a set of files. The agents skipped files in 67.9% of runs. In 80.4% of those runs they either claimed a complete review or left the gap unmentioned. The authors’ conclusion: “agents’ final responses are not reliable accounts of their actions.” It’s a preprint, and the tasks were reviews rather than code changes, but I’ve had summaries like that from our own agents.

METR asked maintainers of scikit-learn, Sphinx and pytest to review agent PRs that had passed SWE-bench’s tests, and roughly half would not have been merged. That’s 296 PRs from agents up to late 2025, with no chance to iterate. Anthropic’s Claude Opus 5 system card now tracks a metric called “false completion claims: claiming a task is complete, successful, or verified when it is not.” That’s the vendor reporting on its own model.

The other half of the problem is us. Steven Shaw and Gideon Nave at Wharton found that when people consulted an AI and the AI was wrong, they went along with it in 73.2% of those trials. It’s a preprint, and those were reasoning puzzles rather than code reviews, but the pull is the same one: a confident summary is easy to accept. They call it cognitive surrender.

Where it showed up for us

Each of these bugs sat where the agent’s own checks couldn’t reach.

Three more from the same month, all on one project.

Green suite, broken field. I typed a 9 into a money field. The app tacked on “.00”, and from the next keystroke on the whole flow was buggy. Inside a new dialog layer, the field re-rendered one pass before its parent caught up, took my keystroke for an outside edit and reset itself. The agent’s explanation: “Jest batches the same way, which is why the suite stayed green while the device broke.”

Proof on the wrong data. Asked to verify a fix end to end, an agent pointed seeded data at the test users and raised a question from the reviewer portal. The tablet showed “Need fix 0”. The app only lists applications it created itself, so the seed proved nothing. When the agent drove the real path instead, it found a bug in the app itself: a reviewer could ask the field officer about a field the officer had no way to answer.

Not finished until it lands. I told the agent wiring up in-app feedback that it wasn’t finished until I saw a report from the tablet land in our feedback inbox. It came back with eight checks, all passing on the real device. The one end state that mattered never happened: every upload died at the storage bucket on an invalid signature, and five reports, my original among them, were lost for good.

“Isn’t that what tests are for?”

A fair question. For pure logic, the tests are the proof. If the suite covers a parser or a fee calculation, I don’t need a video of it.

But tests run inside a model of the world, and the bugs above lived outside that model: in a build cache, in render timing on a real device, in a credential on a server. And verification is slow. In one round, four coding agents took between 8 and 37 minutes each. The single lane checking their work on the devices took about 100 minutes. Tan calls verification “the limiting step, or long pole of software development”, and our numbers agree with her.

Verification can also fail outright. One night I left a batch of 18 PRs to 86 agents and asked for every check on the plugged-in tablet. Two of them rebooted it. It has a pattern lock, so nothing got verified on the tablet that night.

Verification is the slow part now, which makes it the part worth engineering.

How our agents verify their work

We treat verification as tooling the agent runs, not instructions it reads.

On 30 September I read Tan’s guide to pstack, whose first part is titled “Verification is all you need”, and asked an agent whether we could apply it to our case. By that evening the issue was filed and the verification skill was merged. Some things that have held up so far:

Give the agent a lever, not a recipe. Tan calls this “Build the Lever”: give agents tools, not just markdown. Ours is a script of about 1,200 lines that drives the tablet app by the IDs on its controls. It signs in, navigates, taps, types Khmer, records video and pulls the device’s database. A second script plays the reviewer in the web portal, with a –dry-run on every command that changes data. An agent runs one command instead of improvising one.

Check the checker first. Before an agent touches the device, doctor asks whether the setup is worth driving. Is the dev server’s file watcher still alive, or is it serving yesterday’s code? Is the API process older than its last build, the tablet’s clock behind (which breaks every call while the app still looks signed in), or the screen locked? Before doctor, an agent proving a change on a device lost most of its time to exactly these: a broken setup, or a device it couldn’t see. Its first run caught two real problems.

doctor checks the setup before an agent drives the device. A locked screen stops the run.

Take the user’s path, and bring back what it changed. The skill’s proof rules: a shortcut that sets up a case proves nothing about the screens it skips. Record the action as well as the final screen, and check the side effect: the row the tablet saved, the state the server holds. Name the device and the commands actually run. If a device check is still owed, the PR says so on its first line.

Keep the maker away from the checker. The agents that write a fix also write its tests and review it. A separate agent drives the devices afterwards and reports pass or fail per item. In today’s round of 13 fixes, the only failure was a “99+” badge clipped by the bottom bar on the small-screen emulator. It took a one-file fix and passed on the re-check.

Turn every miss into a rule. The seeded-data miss became the rule that shortcuts prove nothing. The reboot became “never reboot the tablet”. The money field became a regression test that fails on the old code. Even the map that tells agents how to reach each screen has a test, which fails when the map names a control the app no longer renders. Addy Osmani’s line for this is that “every repeated correction is a missing piece of the harness.”

Make done mean something

Boris Cherny, who created Claude Code, called giving Claude a way to verify its work “probably the most important thing to get great results out of Claude Code” and said it would “2-3x the quality of the final result”. That’s his estimate, not a measurement, and I can’t put a number on it for our project either. What I can tell you is where our bugs were caught: on a device, by an agent that had to show me.

Agents will keep getting faster at writing code. Deciding what counts as proof is still on us, and it’s where I’d spend effort first.

The next time an agent tells you it’s done, ask it the question I asked on 9 September: how did you verify it?

What does your agent show you when it says “done”?