Tokens
What the model actually reads, and why it explains odd behavior.
What you’ll learn
Four ideas, one module
Embeddings and attention
How each token’s meaning is shaped by everything around it.
Context windows
A hard budget. Fitting inside it isn’t the same as being used well.
Token cost
Why every token costs money and time, on every single call.
Applied exercise
Predict the strongest context
Predict which of three differently structured contexts will produce the strongest answer—and explain why.
- A: The whole handbook
- B: Just the question
- C: Curated context
Cheat sheet
Every answer is shaped by what’s in the context: choose it, measure it, pay for it
One page you keep: the questions to ask on every project, the rules of thumb, and a checklist you can use on your own product. All ten come with The Book.
Leave at the door
Five myths this module breaks
| Myth | What’s actually true |
|---|---|
| “The model reads words.” | It reads tokens: pieces of text mapped to numbers. |
| “A bigger context window fixes it.” | Fitting in the window doesn’t mean being used well. |
| “More context means better answers.” | Irrelevant context dilutes the relevant part. |
| “It remembers earlier in the chat.” | The system re-sends the history on every call. |
| “Tokens are just a billing detail.” | They set cost, speed, limits and odd behavior. |
Before you move on
Can you call this one?
A chatbot follows a rule at turn 3 but ignores it by turn 40.
Explain why using tokens, context and attention. What would you change?
How we would answer
- The weights didn’t change. The model didn’t “forget” like a person; its context did.
- Dilution the rule is now a small part of a long context, competing with 40 turns of history.
- Trimming the system may have cut or summarized old messages to fit the window, rule included.
- Conflict later messages may contradict or override the rule.
- Fixes keep the rule in system instructions sent on every call, trim or summarize history, restate key constraints near the question, and test long conversations (Module 09).
Every module ends on a question like this. Join the waitlist →
The Token Decoder
Zoya Kellan
explores the strange universe between human language and the numerical fragments machines actually process.
Episode 02 · How a model sees the world
Included with the course.
Coming soon · an AI character from the show, not a real researcher.
Next · Module 03