A support assistant starts quoting refund terms that changed three months ago. The team swaps in a newer model. Same wrong terms. Someone rewrites the prompt. Still wrong. Two weeks later a developer notices the policy document in the search index was never reindexed. The model was never the problem.
This happens because the word “AI” gets used for the whole product. When something breaks, there is nowhere to point except the model, so that is what gets replaced. A map fixes that. Once you can name the parts, you can name which part failed.
The model is narrow: weights behind an endpoint. Text in, predicted text out. No memory between calls. No access to your data. No ability to do anything unless something else runs the action. Everything else a user experiences was built by people, on purpose.
The model generates. The system does.
Four boxes that feed the model
| Box | What it decides |
|---|---|
| Context | Everything assembled into this one call: instructions, history, retrieved text |
| Knowledge | Facts the model never trained on: your documents, your data, today’s prices |
| Memory and state | What survives between calls, and what a half-finished run carries forward |
| Interface | What a user can ask, what they see, and how a wrong answer looks to them |
Four boxes that decide what happens
| Box | What it decides |
|---|---|
| Tools and actions | Which real systems it can read, and which ones it can change |
| Orchestration | Which steps run, in what order, and whether code or the model chose |
| Guardrails | Permissions, filters, limits and approvals wrapped around the model |
| Evaluation | The repeatable tests that tell you the product got better or worse |
Cost and latency are not a ninth box. Every box adds some. A retrieval step, a second model call, a validation pass, an approval queue: each one buys you something and charges you for it on every request. That trade is easier to argue about when the boxes have names.
Map any product in five minutes
- Pick one real user request and trace it from the interface to the answer, step by step.
- Mark the model call. Every other step on the list belongs to the system.
- Sort each step into the eight boxes. One step can touch two of them.
- For each empty box ask: missing by design, or missing because nobody decided?
- Circle the box most likely to fail first, and name the person who owns it.
Do this on a whiteboard with the people who built the thing. The argument is the point. Most teams discover that two of them believed different things about what is actually sent to the model on each request.
The most common empty box is evaluation. Products ship on the strength of one good demo, and nobody can say whether last week’s prompt change helped. An empty box is not always a bug, but an unnoticed empty box usually is.
ExplAIn is built on this map. The first module draws it, and every module after that goes deep into one box: context, knowledge and memory, tools, orchestration, guardrails and evaluation.