Token probabilities
Each step is a spread of likely next tokens, not a looked-up answer.
What you’ll learn
Four ideas, one module
Sampling and temperature
How one token gets picked, and why the same prompt can differ.
Hallucination
Why fluent, confident text can still be false.
Reasoning and verification
More thinking improves the odds. Checks decide what ships.
Applied exercise
Where is probability acceptable?
Decide where probability is acceptable in six product scenarios and where verification must be deterministic.
- Writing for people
- Data into systems
- Money and actions
Cheat sheet
Treat every output as a likely guess until something outside the model checks it
One page you keep: the questions to ask on every project, the rules of thumb, and a decision guide you can use on your own product. All ten come with The Book.
Leave at the door
Five myths this module breaks
| Myth | What’s actually true |
|---|---|
| “Temperature 0 makes it deterministic.” | It reduces variation. Repeats aren’t guaranteed. |
| “It sounds sure, so it’s right.” | Confident tone is generated, not measured. |
| “Hallucination is a bug they’ll patch.” | It follows from generating plausible text. |
| “Reasoning models don’t hallucinate.” | They think longer and can still be wrong. |
| “Ask it to check itself and you’re safe.” | Self-checks help. Deterministic checks decide. |
Before you move on
Can you call this one?
A billing assistant confidently tells a customer the wrong late fee.
Why could that happen, and what would you change so it can’t reach the customer?
How we would answer
- Mechanism the fee was generated as a likely-looking number, not looked up. Confident tone says nothing about correctness.
- Not a temperature problem: at temperature 0 it could repeat the same wrong number every time.
- Tools fetch the fee from the billing system and let the model only phrase it.
- Guardrails block any amount in the reply that doesn’t match the billing system’s value.
- Evaluation test fee questions before every release and monitor complaints about amounts.
Every module ends on a question like this. Join the waitlist →
The Confidence Hacker
Maya Voss
investigates why machines sound absolutely certain even when they are spectacularly wrong.
Episode 04 · Reasoning, sampling and hallucination
Included with the course.
Coming soon · an AI character from the show, not a real researcher.
Next · Module 05