---
title: "When Zero Findings Means Nothing Was Read"
author: "Harry"
published: 2026-11-02T01:59:23.969Z
category: "Security Testing"
tags: ["ai-sec"]
canonical: "https://fxblog.dev/posts/ai-security-assessment-ai-sec"
---

# When Zero Findings Means Nothing Was Read

## Key takeaways

- A security scan with zero findings means nothing unless the scanner could actually read the target.
- On OWASP Juice Shop, which ships no lockfile, trivy exits 0 with no dependency findings because a version range is not a version.
- ai-sec lets a scanner declare a partial refusal: it still runs, and the one capability it cannot use grades its module blind.
- Every ai-sec run ends with a Could not evaluate block, and a run that completes with a non-empty ledger says so.
- ai-sec scans at T0 by default, sends nothing, and refuses any tier above the one a human wrote into scope.json, with exit code 2.

> **TL;DR** A security scan that comes back with zero findings looks the same whether the scanner read your code or never got the chance. ai-sec is an AI security assessment bundle for Claude Code that pairs every result with what it could not check. Only here: a scanner that ran but could not read, like trivy on a repo with no lockfile, grades its module `blind` instead of clean, and the run says so under `Could not evaluate`.
>
> **Do this:** Run `node bin/sec.mjs doctor` and a T0 scan on a repo you own, and read the `Could not evaluate` block before the findings.

Here is one row from a scan of OWASP Juice Shop, a deliberately vulnerable practice app:

```text
  M2            2/2        0      0         0            0  unmeasured
```

Two engines installed, both credited with M2 (dependency CVEs), and not one finding. The hurried reading is "nothing was found". The true one is "nothing was read".

**This series, in three parts:**

1. **When Zero Findings Means Nothing Was Read** (you are here)
2. **The SQL Injection Sat Below 188 Test-Suite Findings**
3. **Grade Your Security Scanner Against Answers It Didn't Write**

## Why a clean scan can mean nothing

Juice Shop ships no lockfile. Its `package.json` states `"express": "^4.17.0"`, and a range is not a version. trivy will not guess which one is installed, so it has nothing to match against its database. It exits 0 and reports no dependency results, and every layer above it is right to say there were none.

That is the usual failure: scanners present an empty result the same way whether the code is clean or the parser crashed.

> **Warning**
>
> "0 findings" is a claim about the code only if the scanner could read it. Ask which one you are looking at before you quote it.

## Five questions to ask of any security scan

1. **Did every scanner run, and could it read this target?** A missing or half-blind scanner must show up, not vanish.
2. **How sure is each finding?** Severity says how bad. You also need how sure.
3. **Is it the application, or the test suite?**
4. **Who graded the scanner, and against whose answers?**
5. **Who said you may touch this target?**

## How ai-sec works: seven stages, gates in code

ai-sec runs in Claude Code, but the gates are not prompts. A deterministic controller owns them, and the AI explains and judges without sitting in the hot loop.

discover → scope → scan → merge → verify → score → report

```mermaid
flowchart LR
  A[discover<br/>map the surface] --> B[scope<br/>tier, hosts, authorization]
  B --> C[scan<br/>T0 sends nothing]
  C --> D[merge<br/>by CWE family]
  D --> E[verify<br/>not the finder]
  E --> F[score<br/>graded on answers]
  F --> G[report<br/>verdict + Could not evaluate]
  B -. tier above ceiling .-> X[REFUSED, exit 2]
```

Two scanners are built in and need nothing installed: `builtin` reads code, `probe` reads a live target. Semgrep, Trivy, Gitleaks, Nuclei and ZAP plug into the same adapter protocol when they are on PATH. I chose not to write a detection engine; the value is the layer above one.

## One real run: the row that stopped lying

I could have refused trivy outright on a target with no lockfile. On Juice Shop that would have thrown away its misconfiguration and secret checks, which found 94 findings, to report one gap. So trivy's preflight now declares a partial refusal: the scanner runs, and the one capability it cannot use is named.

```text
  M2            2/2        0      0         1            0  blind
         blind spot: trivy could not check dependency CVEs on this target: no lockfile or
         other resolvable manifest was found, so trivy has no package versions to match
         against its database — zero dependency findings here means nothing was read, not
         that nothing is vulnerable (package.json alone states ranges, not versions)
```

No score went up. The same sentence reaches the report:

![The ai-sec showcase's "Never a quiet pass" page: trivy with no lockfile is a partial refusal graded blind, and a Could not evaluate block lists trivy, an unsupported language and module M11](https://fxblog.dev/media/01a199f7-de88-7140-a53f-7312c3ae961e/1600)

_A run that completes with a non-empty ledger says so on its last line_

## Get it, and what it does not do

```sh
git clone https://gitlab.com/agentic-fx/ai-sec.git
cd ai-sec
node bin/sec.mjs doctor
node bin/sec.mjs scan mock --slug demo
```

It needs Node 20+ and nothing else. `mock/` is a read-only app with planted defects. `scan` with no tier is T0: it reads files and sends nothing.

> **Danger**
>
> Authorized targets only: your own code, employer systems with authorization, staging, CTF, deliberately vulnerable practice apps. Never a third party, never production. Going above T0 needs a human to edit `.sec/<slug>/scope.json`; anything else is refused with exit 2.

It is not a pentest and writes no exploits. Business logic (M11) is `no-engine` on purpose, and nobody outside the project has reviewed it yet.

## Try it

```sh
node bin/sec.mjs scan .
node bin/sec.mjs report
```

Read the last block first. On Juice Shop, M2 is now honestly `blind`, but that scan also returned 246 findings, and the SQL injection in the login route was not on the first screen. Part 2, **The SQL Injection Sat Below 188 Test-Suite Findings**, reads that report in the order it was built to be read.

## Read this with your AI

```text
Read the post "When Zero Findings Means Nothing Was Read".
1. Summarise it in 5 bullets for an engineer who reports security scan results.
2. Ask me 3 questions to check I understood; wait, then correct me.
3. For one scanner I run today, list what it would report if it could not read my code.
```

> **Success**
>
> You can ask whether a zero was read or only reported; in ai-sec a scanner that could not read grades its module `blind`. Next: **The SQL Injection Sat Below 188 Test-Suite Findings**.

## FAQ

### Does ai-sec need scanners installed?

No. It needs Node 20 or later. Two scanners are built in, builtin for code and probe for a live target; Semgrep, Trivy, Gitleaks, Nuclei and ZAP are used when they are on PATH, and doctor says what each absence costs.

### Can I point ai-sec at any website?

No. Only authorized targets: your own code, employer systems with authorization, staging, CTF and deliberately vulnerable practice apps. A host outside the allowlist or a tier above the ceiling in scope.json is refused with exit code 2.

### Is ai-sec a pentest?

No. It writes no exploits, business logic is out of reach by design, and nobody outside the project has reviewed it yet.
