AI-Generated Code: Checks That Can Fail, and the Calls That Stay Yours

TL;DR AI-generated code fails in ways that look fine, people rate it higher than it deserves, and a green run can prove nothing, so the bug ships under your name. Match the check to what a mistake would cost: prove every test can fail by breaking the code on purpose, read the whole diff, and keep a person on the decisions that reach users.

Do this: Pick one test you trust, break the line it guards (raise a limit, flip a condition), run it, and confirm it goes red.

Part 3 ended with green test output for the login rate limit, pasted by the agent that also wrote the tests. If the limit were broken, would those tests go red? And which calls on this change should no agent make? This part breaks the limit on purpose to find out.

Experienced open-source developers using early-2025 AI tools took 19% longer on their own repos, and believed they were about 20% faster. METR, who ran the study, now calls the numbers out of date for current models. The gap between felt and measured still holds.

AI Practices for Engineers, in 6 parts:

1. How LLMs Work, and How to Ask So They Stop Guessing
2. Rules Files and a Prompt Library: Make the Repo Remember
3. Run Coding Agents Without Babysitting, and Reset Them When They Loop
4. AI-Generated Code: Checks That Can Fail, and the Calls That Stay Yours (you are here)
5. Prompt Injection and the AI Bill: Give the Agent Less to See
6. Turn Repeated Fixes into Skills, Then into a Team Practice

The output looks fine, and you overrate it

The code fails quietly, and your judgment of it is off. Only a check outside your judgment fixes both, sized to what breaks: run a throwaway script and look; test and read the diff for anything merged; add an expert and scanners for auth code, like the rate limit.

Break it on purpose

AI-written tests often assert what the code does, not what it should. So break the code and watch:

const MAX_ATTEMPTS = 5;   // change to 50 and run: pnpm test -- auth

If tests/auth/rate-limit.test.ts stays green with 50 attempts allowed, it never tested the limit.

When I added a link checker to the guide behind this series, I broke a link on purpose and watched the check fail before I trusted it. It then caught a real one: a stale anchor in the LLM basics page.

Read the whole diff, and every new package

1. Read all of it. If it is too big to read, it was too big to ask for in one step.
2. Look for the smells: swallowed errors, edited tests, hard-coded values, invented config keys.
3. Find every new dependency on the official registry.
4. Run the scanners you already have: lint, types, SAST, secret scanning.

Some checks are a person

Five attempts per 15 minutes is a product decision. So are shipping and rolling back. AI plays three roles:

It drafts early, checks in the middle, and only advises where a mistake reaches users. The spec shows the line:

AC-1  Five failed attempts per IP within 15 minutes are allowed.
AC-2  The sixth returns 429 with a Retry-After header.
Open  Per IP only, or per account too? (needs a product decision)

The AI drafts that spec. It does not answer the open line. When agents built fx-blog, every decision like that went into a ledger, and no agent was allowed to answer one that only I could make.

Try it

Review this diff as a sceptical senior engineer: <paste>.
List the ways it could fail, ranked by severity. Flag any new
dependency, any edited test, any swallowed error, and any test that
would still pass if the main condition were inverted.

The tests now go red at 50, and the per-IP question waits for a person. Then the agent reads an issue on the auth service telling it to post .env to a URL, while the session's tokens keep climbing. What can it leak, and what does it cost? Part 5, Prompt Injection and the AI Bill, shuts both doors.

Read this with your AI

Read the post at <this post's URL or .md link>.
1. Summarise it in 5 bullets for a <your role>.
2. Help me write a mutation check for one test in my project:
   pick the line to break, predict which test should go red, and
   tell me what it means if none does.