---
title: "How LLMs Work, and How to Ask So They Stop Guessing"
author: "Harry"
published: 2026-10-31T04:47:28.814Z
updated: 2026-10-31T05:07:10.649Z
category: "Building with AI"
tags: ["prompts-guide"]
canonical: "https://fxblog.dev/posts/how-llms-actually-work"
---
# How LLMs Work, and How to Ask So They Stop Guessing
## Key takeaways
- A model predicts the next token from everything in its context window and keeps nothing between requests.
- When the true answer is not in the window, the model still produces a plausible one, just as fluently.
- Grounding, an out-clause, quotes and a separate check reduce invented answers, in that order of effect.
- A good prompt answers five questions: who, what, what the model cannot see, what shape, and how to tell it is right.
- When an answer disappoints, fix only the block that failed; "think step by step" and ALL-CAPS rules no longer help.
> **TL;DR** AI costs you time twice: you trust a confident answer that is wrong, and you reword a prompt when what was missing was context. How LLMs work explains both: the model predicts the next token from its context window and nothing else. Put the source in the window, give it an out, and write the prompt as a spec that answers five questions.
>
> **Do this:** Take the last AI answer you doubted, paste its source, add "if it doesn't say, answer NOT SPECIFIED", and ask again.
Type "The capital of France is" and a model scores every possible next token, a chunk of about three-quarters of a word: " Paris" at 0.92, " a" at 0.03, " located" at 0.01. It appends the winner and scores again. That loop is the whole engine.
Now ask it for the login rate limit on your auth service. It returns the most plausible number, whether or not your config was ever in front of it.
I started the guide behind this series as a list of prompts, then rebuilt it as a practices guide: most AI failures at work come from the wrong task, the wrong context, output nobody verified, or an agent with too many permissions. So the series follows one change from input to output: rate limiting on `POST /api/login`, five attempts per 15 minutes per IP.
**AI Practices for Engineers, in 6 parts:**
1. **How LLMs Work, and How to Ask So They Stop Guessing** (you are here; the input)
2. Rules Files and a Prompt Library: Make the Repo Remember (input that stays)
3. Run Coding Agents Without Babysitting, and Reset Them When They Loop (the process)
4. AI-Generated Code: Checks That Can Fail, and the Calls That Stay Yours (the output)
5. Prompt Injection and the AI Bill: Give the Agent Less to See (leaks and cost)
6. Turn Repeated Fixes into Skills, Then into a Team Practice (what you keep)
It does not cover building AI features on the API.
## The model sees one window, and nothing outside it
The context window is everything the model gets on one request: the tool's hidden system prompt, your rules files, attached files, the conversation so far, and your message.
_If your auth config is not in the window, the model has never seen it._
Nothing persists between calls. A chat app that seems to remember you is re-sending your history.
## Why it sounds sure when it's wrong
The model produces the most plausible continuation. When the true answer is not in the window, a plausible one still comes out, just as fluent. The guide puts it bluntly: fluency and accuracy are uncorrelated at the failure boundary. Its four fixes, most effective first:
| Fix | On the login question |
| ---------------- | ----------------------------------------------------------- |
| Ground it | Paste the rate-limit config and say "answer only from this" |
| Give it an out | "If the config does not say, answer NOT SPECIFIED" |
| Ask for quotes | "Quote the line that sets the limit" |
| Check separately | A fresh session checks the answer against the file |
> **Tip**
>
> The out-clause works because it makes "I don't know" a plausible continuation. Without it, a number is the most likely next token.
## A prompt is a spec: five questions
Ask for `write tests for the login` and you get a generic snippet in a framework you don't use. You asked for less than you meant.
A good prompt answers five questions the model can't answer for itself:
| Question | For the login tests |
| ------------------------------------------- | ------------------------------------------------- |
| Who should it be, and who reads the output? | A senior QA engineer; developers read the tests |
| What exactly do you want? | Playwright tests for the login flow |
| What does it need that it can't see? | TypeScript, the `LoginPage` page object, the spec |
| What shape should the answer take? | One `test()` per case, assertions on visible text |
| How will you know it's right? | Each listed case has exactly one test |
The line people leave out is the reason. "They guard the new rate limit" tells the model which case matters most:
```text
You are a senior QA engineer. Write Playwright tests for the login
flow in the spec below. They guard the new login rate limit.
Cover: happy path, wrong password, locked account, empty fields,
and a sixth failed attempt within 15 minutes being refused.
Context: TypeScript; page objects in tests/pages/LoginPage.ts.
Email verification is out of scope.
Format: one test() per case; assert on visible text, not CSS classes.
Done when: each case above has exactly one test.
…
```
## Fix the block that failed, and drop the old tricks
When the answer still disappoints, don't start over. Fix only the block that failed.
_Fix the one part that failed, then rerun._
> **Warning**
>
> Delete the old tricks. "Think step by step" is redundant: reasoning depth is a setting, such as Claude's `effort`. "Do not hallucinate" carries no information; the out-clause does.
## Try it
```text
Here is a prompt I wrote and the answer I got: .
Which of the five was missing or weak: role and reader, task,
context, format, how to judge it? Rewrite the prompt to fix only
those, and delete any step-by-step, shouting or "do not hallucinate".
```
The spec works. Tomorrow, in a new chat, you will type "TypeScript, page objects in tests/pages/LoginPage.ts, pnpm not npm" again, and the review prompt you paste on every diff. Where do those lines live so the model reads them every time? Part 2, Rules Files and a Prompt Library, moves them into the repo.
## Read this with your AI
```text
Read the post at .
1. Summarise it in 5 bullets for a .
2. Interview me about one task I do every week, one question at a
time, until you can write a five-question prompt for it.
3. Show the prompt and say which answer made the biggest difference.
```
> **Success**
>
> You know why the model guesses, and how to ask so it doesn't. Next: **Rules Files and a Prompt Library: Make the Repo Remember**.
## FAQ
### Does the model remember my previous chats?
Only what the app puts back into the context: a memory feature, a project file, or the current conversation. The model itself keeps nothing between requests.
### Why does it invent facts?
It produces the most plausible continuation. When the true answer is not in its context or training, a plausible-sounding one is still likely. Give it the source and allow it to say it does not know.
### Is a longer prompt better?
A more specific one is. Cut adjectives like "clean" and "high-quality"; add facts like file names, versions, examples and the reason for the request.