When a Test Goes Red: Triage, the Heal Guard and Quarantine

TL;DR When an AI agent's test goes red, the cheapest fix is to hide it, and a green run made by editing the test tells everyone the feature works. AI test failure triage in stlc removes that option. Only here: a fix must pass npm run qa:heal-guard, which diffs it against the base and blocks a removed or weakened assertion, a skip or a retry, and writes its verdict to reports/heal-guard/.

Do this: Before you fix your next red test, label it FUNCTIONAL, SCRIPT, SETUP/ENV or FLAKY, and write down the evidence for that label.

Part 2 ended with ten scripts for orders checkout. The first uat run came back red: TC-INT-ORDERS-001 expected a tax_total of 4.20 and got null. At 2 a.m. the agent has three cheap options: loosen the assertion so null passes, add a retry, or skip the test. Any of them turns the run green and hides what is wrong.

This series, in four parts:

1. A QA Pipeline Your AI Agent Can't Fake
2. Your First stlc Run: Checkout From Docs to Scripts
3. When a Test Goes Red: Triage, the Heal Guard and Quarantine (you are here)
4. The QA Board and the Release Gate

A red test gets a label, not a fix

/qa-run never edits a test or the product. It writes each failure to failures.md with the evidence and exactly one label:

Checkout's failure was SETUP/ENV: the uat seed predates the tax migration. The product was fine, and loosening the assertion would have buried that. The other red test, TC-E2E-ORDERS-001, is FUNCTIONAL. The new order never appears in the list, so it goes to /qa-defect as DEF-orders-001.

FLAKY needs a re-run someone saw pass. I hold that line even for CI: a retry there is recorded as retried-in-ci, never FLAKY, because nobody ran the one immediate, unchanged re-run the label means.

The heal guard: what a fix may not do

For a SCRIPT failure, /qa-heal repairs the test, and npm run qa:heal-guard checks the diff against the base. /qa-heal and /qa-flaky are not done until it exits 0. You can run it yourself on any diff.

• removed, weakened and suppressed (skip, only, retries:, toPass()) changes are blocked, always;
• a changed expected value passes only if every new value is already in the case, or a named person approves it with --human "<quote>".

So changing 4.20 to null, the move an agent reaches for first, is blocked, and the verdict lands in reports/heal-guard/ either way. The guard's own tests try 20 ways to fake a green heal, and each one must be blocked.

Flaky tests are quarantined in the open

TC-E2E-ORDERS-002, the order list filter, fails on CI when the filter chips render late. A re-run passes, so the case is quarantined in the ledger (shortened here: the real one also has Evidence and Released columns):

| Case ID           | Since      | Until      | Reason                                  | Ticket   |
| TC-E2E-ORDERS-002 | 2026-10-05 | 2026-10-30 | filter chips render late on CI (timing) | TASK-201 |

The grant lasts at most 30 days, lint fails once it expires unreleased, and the case still runs and reports its true status. Checkout's second quarantine, TC-API-ORDERS-002, ran out on 2026-10-10 without a release. Lint flags it.

A defect closes on evidence

The rounding bug from part 1 has a file now: /qa-escape opened DEF-orders-002 with Origin: ESCAPED. Its escape analysis names the phase that missed it and the type that should have caught it (in this example, DESIGN and unit).

1. Backfill: design adds a unit case for a three-line order.
2. Verify: /qa-defect orders --verify looks for a run newer than every recurrence in which that case passed.
3. Close: only then does the file get Resolution: fixed (verified <run-id>).

Nobody gets to call it fixed on their word. A repeat failure re-opens the same file.

Try it

Here is a failing Playwright test and its error. Before changing anything, label the failure FUNCTIONAL, SCRIPT, SETUP/ENV or FLAKY, and show the evidence for the label. If it is SCRIPT, propose a fix that keeps every assertion at least as strict as before, and say which line a heal guard would block if you loosened it.

Checkout now has two red tests with known causes, two quarantines (one expired) and two open defects, all in files and none easy to see at a glance. May checkout ship? Part 4, The QA Board and the Release Gate, opens the board on this exact state.

Read this with your AI

Read the post "When a Test Goes Red: Triage, the Heal Guard and Quarantine".
1. Summarise it in 5 bullets for a QA engineer.
2. Ask me 3 questions to check I understood; wait, then correct me.
3. Look at my last three test fixes and say which ones a heal guard would have blocked.