Skip to content
fx-blog

Turn Repeated Fixes into Skills, Then into a Team Practice

A correction typed into every session should become something that runs without you. A skill for API tests with a small eval and five pillars, a one-page team policy measured against a baseline, and four levels to place yourself.

Harry4 min readAdvanced
  • Memory
MarkdownSave

Key takeaways

  • A fix you repeat should become a rule, a hook, a skill or an eval rather than a sentence you type again.
  • Write a skill only when the task repeats, has a house method, and can be checked.
  • Score each skill version on a small eval of real inputs; a change that lowers the score does not ship.
  • AI amplifies a team's existing strengths and problems; in DORA's 2024 data more adoption went with a 7.2% drop in stability.
  • Measure delivery and quality against a baseline, not usage or how fast people feel.
Contents

TL;DR A correction you type into every session lives in your head, costs tokens each time, and leaves with you; a team that adopts AI without a baseline can lose stability and never see it. Turn each repeated fix into AI agent skills, rules or hooks that run without you, prove them with an eval held to five pillars, then share them under a one-page policy and measure outcomes.

Do this: Write down the correction you typed most this week, and decide whether it is a rules line, a skill or a hook.

Part 5 ended on a correction you keep paying for: "reuse the fixtures, don't create a raw HTTP client", typed into every fresh session that writes the login API tests, by you and by your teammates. This part makes the repo learn it once, and the team with it.

By the tenth time, the problem isn't the agent. The fix lives in your head instead of in the repo, so every new session, and every teammate, starts without it.

AI Practices for Engineers, in 6 parts:

  1. How LLMs Work, and How to Ask So They Stop Guessing
  2. Rules Files and a Prompt Library: Make the Repo Remember
  3. Run Coding Agents Without Babysitting, and Reset Them When They Loop
  4. AI-Generated Code: Checks That Can Fail, and the Calls That Stay Yours
  5. Prompt Injection and the AI Bill: Give the Agent Less to See
  6. Turn Repeated Fixes into Skills, Then into a Team Practice (you are here)

Where a repeated fix belongs

Where a repeated fix should live: a fact for every task goes in the rules file, a method in a skill, a fixed prompt in a slash command, a must-always rule in a hook, and proof in an eval

The fixture rule is a method for one kind of task, writing API tests, so it belongs in a skill. A skill loads only when the task calls for it, which keeps it out of the AGENTS.md you trimmed in part 2.

A skill, proven by an eval

markdown.claude/skills/gen-api-tests/SKILL.md
---
name: gen-api-tests
description: Generate Playwright API specs from case files under cases/api/.
  Use when asked to write, generate or automate API tests.
---
## Steps
1. Read the case file and reference/case-format.md.
2. Copy templates/spec.ts.tmpl; fill one test() per case.
3. Reuse fixtures from tests/fixtures/api.ts. Never create a raw HTTP client.
4. Run scripts/check-cases.mjs <spec>; fix until it exits 0.
## Done when
- The check exits 0 and every case ID is in exactly one test title.

Point it at cases/api/auth/login-rate-limit.md and step 3 is applied every time, by anyone. The description decides whether it loads at all, so write triggers, not a tagline.

Prove it with five to ten real inputs, including a missing case file and a request that must not trigger it. Score each version as cases passed over total; a change that lowers the score doesn't ship. Then hold it to the five pillars I use for every agentic-fx project, each with a measure that can go red: intelligence, memory and learning, judgment and integrity, verification and safety, value and experience.

From you to the team

In 2024, Google's DORA research found that a 25% rise in AI adoption went with a 7.2% drop in delivery stability, alongside gains in documentation and review speed. Their 2025 summary: "AI doesn't fix a team; it amplifies what's already there."

So the skill only helps if the team shares it, and a teammate who pastes the login code into a personal account needs a written rule. Start with one page:

textteam-ai-policy.md (auth team, v1)
Tools:     business tier only; personal accounts not for company code.
Data:      no secrets, credentials or PII in prompts; agents can't read .env*.
Ownership: the person who commits owns the code, whoever wrote it.
Review:    every diff read by a human before merge.
Agents:    branches only; push, deploy and delete need approval.
Config:    AGENTS.md and skills live in the repo, owned by <name>.

Where you go from here

The login rate limit went from a number the model invented to a spec, a fenced agent run, a test proven to fail, a skill and a team policy. The guide's four levels place you on that path:

Level Checkpoint
1 User Faced with a bad answer, you name which block failed instead of retrying
2 Practitioner A teammate gets on-convention output from their first prompt
3 Engineer You can say with data whether a prompt change helped, and what a request costs
4 Master You package an expert procedure as a skill a junior engineer can run

Move up when the checkpoint feels easy. Level 3 is the API, which this series did not teach.

Try it

textPaste into your AI
Here is a task I keep asking you to do: <describe, plus 2 past prompts>.
Should it be a rules-file line, a skill, a slash command or a hook?
If a skill: draft SKILL.md with Steps, Done when (a check that can
fail) and 4 eval cases, including one request that must NOT trigger it.

Read this with your AI

textPaste into your AI
Read the post at <this post's URL or .md link>.
Act as my coach for four weeks: place me on the four levels, set one
exercise per week, and at the end of each week check me against
that level's checkpoint.

FAQ

Skill, rules file or slash command?

A rules file for facts needed on every task, a skill for a method for one kind of task, a slash command for a fixed prompt with arguments, and a hook for anything that must always happen.

What should a team AI policy cover first?

Approved tools on a business tier, data that never goes into prompts, who owns AI-written code, review rules, and limits on autonomous agents.

Do I need the API to get good at AI?

Not for levels one and two. Level three is for people building features or pipelines on top of a model.

Sources

  1. AI Practices Guide: Designing skills
  2. AI Practices Guide: The five pillars
  3. AI Practices Guide: AI for teams
  4. AI Practices Guide: Mastery roadmap
  5. Google Cloud: DORA 2024 highlights
  6. Google: DORA 2025 report

Was this useful?

Only you and the owner see your answer and rating. Send private feedback

Enjoyed it? Support fx-blog with a coffee.

Comments

Oldest first

No comments yet. Start the conversation.

You sign in first; what you typed stays here. · 0 / 2,000