---
title: "Write the Storefront Journey, Then Read the Verdict"
author: "Harry"
published: 2026-11-08T13:23:39.813Z
category: "Performance Testing"
tags: ["surge"]
canonical: "https://fxblog.dev/posts/yaml-load-test-scenario"
---

# Write the Storefront Journey, Then Read the Verdict

## Key takeaways

- Write the thresholds before the journey; a scenario with no thresholds produces a pass that only means the run finished.
- check\_rate is the threshold people leave out, and without it a run where every request returns 200 with the wrong body is green.
- storefront passed p95 at 41.5 ms and still failed: error\_rate 27.16 % against a 1 % threshold, check\_rate 80.32 % against 99 %.
- A ramp stage exists so you do not measure your connection pool filling up; starting 30 users at once measures the pool.
- Think time written as a range, 150ms..400ms, stops 30 users marching in step and producing a throughput sawtooth that is an artefact of the scenario.
- surge run exits non-zero when a threshold breaks, so a CI gate needs no output parsing.
- Read a report in this order: the verdict, which threshold broke and by how much, the warnings about whether the run was worth anything, then the failures.

> **TL;DR** A yaml load test scenario that cannot fail is not a test. Write the thresholds before the journey, and the run ends with an exit code CI can act on: storefront's did, at FAIL, 2 of 3 thresholds breached, 979 of 3,604 requests failed.
>
> **Do this:** Open your own scenario and look for the thresholds. If there are none, your last green run meant "it finished".

Both runs in [part 2](https://fxblog.dev/posts/open-vs-closed-workload-model) printed `PASS` with a warning attached: no thresholds were set, so neither had any way to fail. This is the file that fixes it.

I write the thresholds first, before a single request. It feels backwards for about a minute, and then it stops you from writing a test that agrees with whatever happens.

1. Your Load Test Went Green. What Did It Measure?
2. Closed Loops Hide the Cliff. Open Arrivals Find It.
3. **Write the Storefront Journey, Then Read the Verdict**
4. Several Machines, One p99 You Can Defend

[Write the Storefront Journey, Then Read the Verdict](https://www.youtube.com/watch?v=ZKPlVfrvjrg)

## Start with what you are willing to promise

A threshold is the sentence you would say in a review. Write those three lines first and the rest of the file has something to be measured against.

```yaml
name: storefront
type: load
base_url: http://localhost:18080

thresholds:
  - p95 < 150ms
  - error_rate < 1%
  - check_rate > 99%
```

> **Tip**
>
> `check_rate` is the one people leave out. Without it a run where every request returns 200 with the wrong body is a pass.

## Say how much load, and in what shape

storefront ramps to 30 users over five seconds and holds them for twenty-five. The ramp is not decoration: starting 30 users at once measures your connection pool filling up.

```yaml
load:
  stages:
    - { duration: 5s, target: 30 }
    - { duration: 25s, target: 30 }
```

Part 2 is the decision hiding in that block. No `model` means closed, so these numbers are about 30 users, not about arrivals.

## Write the journey as the user walks it

Four steps, each with a name, a request, a check, and think time. The names are what you will read in the report, so name them after the user's intent.

```yaml
flow:
  - name: health
    get: /health
    check:
      status: 200
      json: { status: ok }
    think: 150ms..400ms

  - name: catalogue
    get: /flaky
    check:
      status: 200
      json: { status: ok }
    think: 150ms..400ms

  - name: account
    get: /api/items
    check:
      status: 200
    think: 150ms..400ms
```

> **Note**
>
> `think: 150ms..400ms` is a range, picked per iteration. Fixed think time makes 30 users march in step and produces a throughput sawtooth that is an artefact of your scenario, not of the system.

## Run it, and read four things

```sh
surge run fixtures/storefront.yaml -html report.html
```

The console answers in the order you need it:

```text
thresholds
  ✓ p95 < 150ms                        (actual 41.5ms)
  ✗ error_rate < 1%                    (actual 27.16%)
  ✗ check_rate > 99%                   (actual 80.32%)

FAIL  2 of 3 thresholds breached · 3,604 requests · 979 failed (27.16%) · 1,073 checks failed
```

Read it in this order: **the verdict**, then **which threshold broke and by how much**, then **the warnings about whether the run was worth anything**, then the failures themselves.

![The Overview tab of storefront's report: a FAIL badge reading two of three thresholds breached, tiles for requests, failed, error rate and p95, and each threshold with the value measured against it](https://fxblog.dev/media/01a1bb52-e2f8-77dc-b230-a408f99c3248/1600)

_p95 passed at 41.5 ms. It was never the problem, and a report that led with it would have hidden the two that were._

> **Warning**
>
> p95 passed. If your gate watches only latency, this run is green and 979 requests failed.

## The exit code is the point

`surge run` exits non-zero when a threshold breaks, so a CI job needs no output parsing:

```sh
surge run fixtures/storefront.yaml -html report.html || echo "gate failed: $?"
```

Keep `report.html` as a build artefact. It is one self-contained file with the failures and their bodies in it, which is what you will want in three weeks when someone asks what happened.

## Try it

```sh
go run ./examples/target -addr :18080 -latency 40ms -fail 12 &
surge run fixtures/storefront.yaml -html report.html
open report.html
```

storefront now fails honestly, from one machine, at 30 users. The generator sat at 1 % CPU, so nothing here was limited by the laptop — but raise the load far enough and it will be, and then the number describes the laptop again, exactly as part 1 warned. Part 4 adds machines, and answers the question that comes with them: if three agents each report a p99, which of them is the p99?

## Read this with your AI

```text
Read the post at https://fxblog.dev/posts/yaml-load-test-scenario.
1. Summarise the four blocks of a surge scenario in 5 bullets.
2. Look at my own load test and tell me which thresholds it is missing.
3. Write me a scenario for one journey through my API, thresholds first.
```

> **Success**
>
> You can now write a scenario whose run has a way to fail, and read its verdict in the order that matters. Every field a scenario can carry is in the [scenario reference](https://surge.fxblog.dev/docs/stable/scenario-reference) at [surge.fxblog.dev](https://surge.fxblog.dev/). Next: **Several Machines, One p99 You Can Defend**.

## FAQ

### Which thresholds should I start with?

One latency percentile, one error rate and one check rate. The latency one is the number people quote, the error rate catches transport failures, and the check rate catches a server that answers 200 with the wrong body. storefront uses p95 \< 150ms, error\_rate \< 1% and check\_rate \> 99%.

### Why did p95 pass while the run failed?

Because latency was never the problem. The server answered quickly and answered wrongly: 27.16 % of requests failed and 1,073 checks did not hold. A gate that watches only latency would have called this run green.

### Do I need the demo target to follow along?

No, but it makes the failures reproducible. examples/target in the surge repository serves the endpoints storefront uses and injects failures on demand with -fail, so the numbers in this post come out of a run you can repeat rather than out of a description.

### Can I write the scenario in Go instead of YAML?

Yes. A scenario is a program, so anything you can call from Go can be measured: gRPC, SQL, a message queue, a vendor SDK. YAML covers HTTP and WebSocket, which is most of what people need; the Go library page covers the rest.
