13 agents, 0 errors.
Green but empty
- Agent said
- 13 agents, 0 errors.
- Actually
- Three defects were still there. One was silent data loss on notes past a 256 KB limit.
- Caught by
- A final review, and not trusting the green
Zero errors means nothing crashed. It does not mean the work is right.
I had 13 agents rebuild sync for my notes app and it came back with 0 errors. 1,364 tests passing, lint clean, builds green, so by every number I had it was done.
It wasn’t. The last review found that a note over 256 KB would get dropped from the list sync reads, and sync treats a missing note as a deleted one. So it would have wiped that note from the server and every device, the one job sync has lol
No test tried a note that big, and the review runs last so there was no agent left to fix it. “0 errors” was true, it just didn’t mean what I wanted it to mean.
So now agent reports are claims for me, not proof. My main session reruns the checks and reads the last review before anything counts as done, you can try the same thing:
Don't trust the agents' summaries. Run the checks yourself, read the final review, and list anything still unfixed. For each item, tell me what breaks for a user.PS. this was only the worst one, there were 3 defects left after that run