Skip to content
fx-blog

Will AI Replace Testers? Make Your Judgment the Evidence

AI drafts test cases, code and bug summaries in seconds, which moves a tester's value from producing artifacts to judging them. Review AI output like a junior's work, refuse unverifiable claims, and show your judgment through a portfolio, outcome-based CV bullets and a yearly career check.

Harry5 min readIntermediate
  • Judgment
MarkdownSave

Key takeaways

  • AI tools draft test cases, test code and bug summaries in seconds, which moves a tester's value from producing those artifacts to judging them.
  • A generated test that runs green but asserts nothing is worse than no test, because it adds false confidence.
  • Agentic test tools are for finding things; merge gates should stay on deterministic tests that fail the same way twice.
  • A tester's portfolio should show judgment through real documents such as test designs, risk tables, release memos and design docs, never an employer's internal material.
  • CV bullets that state what changed because you were there, with numbers you can defend, stand out from bullets that list duties.
Contents

TL;DR AI now drafts test cases, test code and bug summaries in seconds, so a tester valued for producing those artifacts is easy to replace, and a green generated test that asserts nothing adds false confidence. Will AI replace testers? The repo's answer: it moves the value from producing artifacts to judging them. Judgment is also what gets you hired, so make it visible in your reviews, your portfolio and your CV.

Do this: Rewrite three bullets on your CV so each one says what changed because you were there, using only numbers you could defend in an interview.

Part 6 ended with the SDET funded and a vendor demo: an agent that "tested all of FieldFlow in 20 minutes, 0 failures". If tests are that cheap to produce, what is a tester's work worth, and how do you show it to the next employer?

Zero failures in 20 minutes is a claim about the agent. Whether it says anything about FieldFlow is the question a tester is paid to ask.

QA Career Path, in seven parts:

  1. The QA Career Path: Map Yourself by Scope, Not by Title
  2. New QA Engineer? Test the Risk, Not the Ticket
  3. Manual to Automation Testing in Six Months: Skip the Tutorial Loop
  4. Senior QA: Depth Everywhere Is Depth Nowhere
  5. After Senior QA: Lead the Release or Own the Platform?
  6. QA Manager to Head of QA: The Escape Bill That Wins Budget
  7. Will AI Replace Testers? Make Your Judgment the Evidence (you are here)

What AI takes, and the catch you keep

AI output is a draft with confidence, and QA's trade is refusing to confuse confidence with evidence.

Task Good use The catch you own
Draft test cases Obvious partitions and boundaries, fast A third is noise; the domain cases (offline sync, the VAT edge) are missing
Write automated tests Scaffolding, locators, data builders A test that runs green but asserts nothing
Summarise logs A fast hypothesis Check the raw log line before routing the bug

Four rules apply at every level:

  1. No production or customer data in tools you don't control.
  2. You sign what you submit, whoever drafted it.
  3. A test you can't explain is a liability: if you can't say what would make it fail, it doesn't merge.
  4. AI never decides severity, risk depth or go/no-go.

The vendor demo, checked

An agentic tool drives a real browser from a plain-language goal. Three rules answer the demo:

  • Gate on deterministic tests; use agents to find things. A merge gate must fail the same way twice, and an agent may "succeed" by a path no user would take.
  • Sandbox it like a new contractor. Test accounts only. An agent that can click "send invoice" will.
  • Read the transcript, not the verdict. A PASS after 40 steps through the settings page is a finding about the agent.

Your judgment, made visible

Your best work is the escape that never happened, and nobody sees that. Documents that show judgment do; the exercises this series followed already are a portfolio.

A portfolio built from the FieldFlow exercises, one document per level

On a CV, "executed test cases" describes every tester. The repo's examples:

  • ❌ "Responsible for regression testing."
  • ✅ "Cut regression from 7 to 2 tester-days by moving 40 checks to API-level tests in CI."

One new door, and a yearly check

AI also opened a role: the AI quality or eval engineer, who builds eval sets, calibrates LLM judges and watches for drift after model updates. Its core is test design, oracles and refusing a confident report, plus enough Python to run an eval harness.

Whatever the move (up, sideways into development, SRE or product, or out and back), answer four questions once a year, in writing:

  1. What can I do now that I couldn't a year ago?
  2. Which work energised me, and which drained me?
  3. If I stay two more years, where will I be, and do I want that?
  4. What one move could I test in a small way next quarter?

A brag document kept all year (problems solved, results, feedback) answers them. And if every release needs heroics, the process is broken, not you.

Try it

textPaste into your AI
Here are three bullets from my CV: <paste>.
Rewrite each as an outcome: what changed because I was there.
Ask me for the before and after numbers instead of inventing them,
and mark any bullet I can't back with evidence.

How to check what AI writes is in AI in Testing: Use It Without Trusting It, and Test AI Features, the last part of the QA Skills Path.

Seven parts ago you placed yourself on the ladder by scope. The repo's exercises are there for your next move.

Read this with your AI

textPaste into your AI
Read the post "Will AI Replace Testers? Make Your Judgment the Evidence".
1. Summarise it in 5 bullets for a tester worried about AI.
2. Ask me 3 questions to check I understood; wait, then correct me.
3. Pick one FieldFlow exercise from the series for my level and help me start it.

FAQ

Will AI replace software testers?

The source's view is that AI takes over producing drafts (test cases, test code, summaries) and moves the tester's value to judging them: what would make a test fail, whether the evidence holds, and which risks matter. Automation does not replace testers; it frees them for the testing only humans do.

What is an AI eval engineer?

A role in companies that ship LLM features: building and maintaining eval sets, calibrating LLM judges against human labels, running adversarial passes and watching for drift after model updates. Test design, oracles and risk thinking transfer directly.

What should a QA portfolio contain?

Documents that show judgment for your level: test design and reproducible bug reports for an engineer, a risk-based plan and a readable automation project for a senior, a release plan, crisis memo and sign-off for a lead, a cross-team design doc for staff.

Sources

  1. QA Career Path repo: AI in QA (docs/12)
  2. QA Career Path repo: portfolio, CV and job search (docs/16)
  3. QA Career Path repo: career moves and sustainability (docs/18)
  4. Playwright MCP server

Was this useful?

Only you and the owner see your answer and rating. Send private feedback

Enjoyed it? Support fx-blog with a coffee.

Senior QA: Depth Everywhere Is Depth Nowhere

A senior QA engineer who tests everything equally leaves the risky areas thin. Score each area by impact times likelihood, test the top deepest, tell the PM in writing what gets only a smoke pass, and make every production escape leave a regression test behind.

4 min readIntermediate

Comments

Oldest first

No comments yet. Start the conversation.

You sign in first; what you typed stays here. · 0 / 2,000