Have Tests Verify the Work

The agent announces that the bug is fixed and adds a test. The test passes. You close the ticket. Two weeks later, the same incident comes back in production, identical to the first. Going back over the diff, you find that the added test never went through the affected code path: it checked that a neighboring function did return a dictionary. It would have passed before the fix exactly as it does after. You never had proof that anything was repaired, only a green light produced at the same time as a change.

This is the most misunderstood point about working with an agent. A test written after a fix, by the one who just made the fix, doesn't test the bug: it tests the code as it was just written. The agent's reasoning is circular, and it is so without any intent to deceive. It observes the current behavior of its own code and records it as the expected behavior. A test whose failure you've never seen proves nothing at all. The discipline that solves this problem is an old one and fits in one sentence: red before green. With an agent, its status changes: from an optional good practice, it becomes the only way to know whether the fix has anything to do with the problem.