Skip to content
fx-blog

Your First stlc Run: Checkout From Docs to Scripts

You cannot tell whether AI-written tests cover the spec. The first stlc run on orders checkout: requirement ids, a traceability table created with zero rows where each column has one writing step, a case review, and ten scripts.

Harry4 min readIntermediate
  • Intelligence
  • Verification
MarkdownSave

Key takeaways

  • stlc starts from what the product should do: analyze turns a spec into requirement ids with a source line each.
  • Planning creates the traceability table with zero rows, so nobody can pre-fill a coverage number.
  • Each column of the table has one writing step: design appends rows, script flips Scripted, the run fills Status, and reports only read.
  • A read-only case review returns REVISE FIRST on any High finding, and scripting will not start until it is fixed.
  • Scripts mirror cases one to one: lint fails a script with no case, and a case with no script keeps a red cross in Scripted.
Contents

TL;DR You cannot tell whether your AI-written tests cover the spec, so coverage is whatever the agent says. In stlc the traceability table is created with zero rows and each column has exactly one writing step, which makes AI-generated tests you can trust: a gap shows as a gap, not as a number.

Do this: Paste one feature's spec into your AI and ask for a requirement register: one row per verifiable behaviour, with an id and source line.

Part 1 left orders checkout with forty green tests and no three-line order. Now checkout goes through stlc from its spec, and the traceability table fills column by column, each column written by one step. By the end every case is tied to a requirement, and the cases still without a script show ❌.

This series, in four parts:

  1. A QA Pipeline Your AI Agent Can't Fake
  2. Your First stlc Run: Checkout From Docs to Scripts (you are here)
  3. When a Test Goes Red: Triage, the Heal Guard and Quarantine
  4. The QA Board and the Release Gate

Start from the spec, not the code

/qa-analyze orders docs/orders-spec.md writes one requirement per verifiable behaviour into the analysis, each with a source anchor you can open. In the sample, REQ-001 covers creating an order and its totals, and REQ-005 speed under load. Tests written from what the code does only prove the code agrees with itself.

One empty table, one writer per column

/qa-plan orders creates the table every later step writes into. It starts with zero rows, so there is no coverage number to pre-fill.

textplans/orders/2026-10-01-test-plan.md
| Req | Case ID | Type | Module | Priority | Scripted | Last run | Status | Report |
  • design appends rows;
  • script flips Scripted from ❌ to ✅;
  • run fills Last run, Status and Report;
  • reports and gates only read.

The contract names that one writer per column (CONTRACTS §3). A hook runs npm run qa:lint after every file the agent writes, so a wrong header, a Status outside ✅ ❌ ⛔ ⊘ —, or a row with no case file is rejected on the spot. npm run qa:coverage counts requirements against cases and lists any requirement with no case as a gap, never as 100 %.

Catch vague cases before they are automated

/qa-design orders --type all writes twelve cases with observable expected values, such as TC-SEC-ORDERS-001: a Viewer who creates an order gets a 403. Then /qa-review orders reads them before any automation. Any High finding returns REVISE FIRST, and /qa-script will not start while that review is the latest on file.

The case review before scripting: twelve checkout cases go to /qa-review; a High finding means REVISE FIRST and the cases go back to design; otherwise READY FOR /qa-script

My own lint missed vague cases at first. Before 2.0, qa:lint checked only a case's shape, and a one-step e2e case with an empty Expected Result passed with 0 errors. Now every step needs an Expected, and wording like "works" or "as expected" draws a warning.

One script per case

/qa-script orders --type all writes specs that mirror the cases: TC-API-ORDERS-001 becomes scripts/api/orders/TC-API-ORDERS-001-x.spec.ts. Lint fails any script with no case behind it, and a case with no script keeps ❌ in Scripted. npm run heal:check also rejects sleeps, skip, only, conditionals inside a test and tests that assert nothing.

Ten of twelve are scripted. Security and performance still show ❌.

The stlc sidebar for orders checkout: the summary, and the next actions, each with its command to copy

Try it

textPaste into your AI
Here is the spec for one feature of my product. Write its requirement register: one row per verifiable behaviour, with an id, the statement, the module and the line of the spec it came from. Then list every ambiguity as a question with a proposed default.

Checkout has twelve rows, ten scripts and no failure yet. The first uat run is about to come back red on TC-INT-ORDERS-001, expecting a tax_total of 4.20 and getting null, and the agent could make that green in one line. Part 3, When a Test Goes Red: Triage, the Heal Guard and Quarantine, shows what stlc lets it do instead.

Read this with your AI

textPaste into your AI
Read the post "Your First stlc Run: Checkout From Docs to Scripts".
1. Summarise it in 5 bullets for a QA engineer.
2. Ask me 3 questions to check I understood; wait, then correct me.
3. Look at my current test suite and say which tests would have no row in a traceability table, and why.

FAQ

What does stlc need before the first run?

One install command, npx agentic-fx@latest install --bundle qa, then PROJECT-PROFILE.md filled in: the app's URLs and OpenAPI spec, the enabled test types, environments, browsers and quality gates.

Can the agent pre-fill the traceability table to look complete?

Planning creates it with zero rows and the contract gives each column one writer. A hook runs npm run qa:lint after every file the agent writes, and it rejects a wrong header, a Status outside the vocabulary or a row with no case file. npm run qa:coverage lists any requirement with no case as a gap.

Sources

  1. stlc README: install and configure
  2. stlc CONTRACTS.md §3: the traceability table

Was this useful?

Only you and the owner see your answer and rating. Send private feedback

Enjoyed it? Support fx-blog with a coffee.

The QA Board and the Release Gate

"May it ship?" should not be answered in a chat. The stlc board on orders checkout, and a QA release gate: a report verdict derived from its gate rows, rejected by lint if it disagrees, which the dev deploy step reads before checkout ships.

4 min readAdvanced

A QA Pipeline Your AI Agent Can't Fake

Forty green AI-written tests shipped a bug and nobody could say what they proved. Five questions to ask any AI test pipeline, and the route stlc takes from docs to a verdict, where a feature's state is computed from the files each step wrote.

4 min readBeginner

Comments

Oldest first

No comments yet. Start the conversation.

You sign in first; what you typed stays here. · 0 / 2,000