Three different problems
Information, state and behavior each need a different mechanism.
What you’ll learn
Four ideas, one module.
Semantic search
Embeddings, vector similarity and chunking, without the math.
RAG and grounding
Find the right text, then answer from it, with sources.
Memory and fine-tuning
Where state should live, and when to change the model itself.
Applied exercise
Choose the mechanism
Choose prompt, retrieval, memory, database, fine-tuning—or a combination—for five product scenarios.
- Documents and policies
- Users and live data
- Behavior and style
Cheat sheet · for your projects
Before choosing a mechanism, ask: is this information, state or behavior?
Ask on every project
- Is this information to know, state to track, or behavior to shape?
- How often does this knowledge change, and who updates the source?
- Must the answer be exact and live, or is the closest match good enough?
- Who may see each document or memory, and is that filtered before retrieval?
- When an answer is wrong, can we see which chunks were retrieved?
Rules of thumb
- Retrieve facts that change or need a citation; don’t bake them into the weights.
- Read orders, balances and permissions from the database of record, never from memory.
- Combine meaning-based and keyword search when users look up codes, names or IDs.
- Fine-tune only after prompting falls short and you have labeled examples and an eval set.
Decision guide
Pick the mechanism
- If it’s a stable rule, tone or short style guide→ prompt, with good and bad examples
- If it’s documents that change or need sources→ retrieval with citations and metadata filters
- If it’s per-user preferences or context across sessions→ application memory the user can see
- If it must be exact and live (orders, balances, permissions)→ database, via code or a tool
- If prompts can’t hold a format or narrow task at your volume and cost→ fine-tune, with evals
- If it spans several→ combine them, but name the primary mechanism and who owns it
Leave at the door
Five myths this module breaks.
| Myth | What’s actually true |
|---|---|
| “Fine-tuning teaches it our documents.” | It shifts behavior. Retrieval supplies the facts. |
| “RAG stops hallucinations.” | Only if retrieval finds the right, current text. |
| “The vector database is the memory.” | It’s one store. Memory is what you choose to keep. |
| “Similar means relevant.” | Close in meaning can still be wrong or outdated. |
| “Big context windows kill retrieval.” | Cost, latency and access still favor selecting. |
Next · Module 06