Opening · 6 min
How old is the maths behind AI?
Write one year: the newest piece of mathematics an assistant needs to answer you.
One guess, on your own
The free deck
Module 00 as slides
The one idea
Almost none of the maths is new.
What changed was compute, data and one architecture.
Before the machines · 13 min
The maths arrives first
| Year | Idea | What it made possible |
|---|---|---|
| 1676 | Chain rule, Leibniz | Derivatives through nested functions |
| 1763 | Bayes, read to the Royal Society | Updating a belief when evidence arrives |
| 1805 | Least squares, Legendre | Fitting a curve to noisy measurements |
| 1847 | Gradient descent, Cauchy | Walking downhill to the best fit |
| 1906 | Markov chains | Predicting the next state from this one |
Before the machines
Training a model is Cauchy, 1847, at scale.
Gradient descent, with the chain rule spreading the blame back across the layers.
Machines that learn · 9 min
The first neural era
1943 · A neuron as logic
McCulloch and Pitts write a brain cell as mathematics.
1956 · The field is named
The Dartmouth workshop coins “artificial intelligence”.
1958 · It learns from data
Rosenblatt’s perceptron adjusts its own weights.
1969 · The limit is proved
Minsky and Papert show a single layer cannot do XOR.
Two winters · 9 min
Then the money left. Twice.
1974–1980
The Lighthill report judged AI had not delivered, and British funding was cut.
1987–1993
The Lisp machine market collapsed as ordinary workstations got cheaper.
What survived both
Backpropagation, published 1970 and popularised in 1986. LSTM in 1997.
Two winters
The ideas were right. The computers were not.
Both collapses followed promises sized for hardware that did not exist.
What actually changed · 11 min
None of it was new maths
Data · 2009
ImageNet: 1.2 million labelled images, far more than anyone had trained on.
Compute · 2012
AlexNet on two GPUs: 15.3% top-five error against 26.2% for second place.
Architecture · 2017
The transformer dropped recurrence, so training could run in parallel.
Scale · 2020 on
Scaling laws showed loss falls predictably as model, data and compute grow.
What actually changed
What was old. What was new.
Old, and still running
17th to 19th century mathematics.
- Chain rule, 1676
- Least squares, 1805
- Gradient descent, 1847
- Markov chains, 1906
New in the 2010s
None of it a new branch of maths.
- The internet as a training set
- GPUs doing the arithmetic
- One architecture: 2017
- Money at industrial scale
The payoff
Compute, data, architecture.
Nothing on the mathematics side had to be invented.
Applied exercise · 7 min
Date the idea behind the feature
Take a feature you use and name the oldest piece of mathematics it depends on, the year it was published, and the gap between that year and the product shipping.
- Streaming suggestions
- Face unlock on a phone
- A chat assistant’s reply
- Groups of 3–4, one feature each
- Name the oldest idea and year: 4 min
- Give the date and the gap: 3 min
Close · 5 min
Five myths to leave at the door
| Myth | What is actually true |
|---|---|
| “AI was invented in the last few years.” | Most of its maths is 17th to 19th century. |
| “Neural networks are a new idea.” | Rosenblatt built a learning machine in 1958. |
| “Hinton invented backpropagation.” | Linnainmaa published it in 1970; 1986 spread it. |
| “AI progress has been steady.” | Funding collapsed twice: 1974 and 1987. |
| “The transformer was a maths leap.” | It made training parallel. Scale did the rest. |
Check your history
A vendor calls their method a brand-new breakthrough.
Name two questions that test the claim.
Show the answer
- Which part is actually new: the mathematics, the architecture, the data or the compute?
- What is the oldest idea it depends on, and what year is that idea from?
- If it is old mathematics at new scale, what changed to make it work now?
- Has this been tried before and abandoned, and what killed it then?
- A good answer names the layer the novelty sits in. Almost always it is data, compute or packaging.
What you’ll learn
Four ideas, one module
How old the maths is
Chain rule 1676. Bayes 1763. Gradient descent 1847. Centuries, not years.
Why it took so long
The ideas arrived first. The compute and the data arrived much later.
The two AI winters
What was promised, what shipped, and why the funding left twice.
What actually changed
GPUs, the internet as a training set, and one architecture: 2017.
Applied exercise
Date the idea behind the feature
Take a feature you use and name the oldest piece of mathematics it depends on, the year it was published, and the gap between that year and the product shipping.
- Streaming suggestions
- Face unlock on a phone
- A chat assistant’s reply
Cheat sheet
Almost none of the maths is new. Compute, data and one architecture changed
One page you keep: the questions to ask on every project, the rules of thumb, and a checklist you can use on your own product. All ten come with The Book.
Leave at the door
Five myths this module breaks
| Myth | What’s actually true |
|---|---|
| “AI was invented in the last few years.” | Most of its maths is 17th to 19th century. |
| “Neural networks are a new idea.” | Rosenblatt built a learning machine in 1958. |
| “Hinton invented backpropagation.” | Linnainmaa published it in 1970; 1986 spread it. |
| “AI progress has been steady.” | Funding collapsed twice: 1974 and 1987. |
| “The transformer was a maths leap.” | It made training parallel. Scale did the rest. |
Before you move on
Can you call this one?
A vendor calls their method a brand-new breakthrough.
Name two questions that test the claim.
How we would answer
- Which part is actually new: the mathematics, the architecture, the data or the compute?
- What is the oldest idea it depends on, and what year is that idea from?
- If it is old mathematics at new scale, what changed to make it work now?
- Has this been tried before and abandoned, and what killed it then?
- A good answer names the layer the novelty sits in. Almost always it is data, compute or packaging.
Every module ends on a question like this. Join the waitlist →
The Algorithm Archaeologist
Simone Vale
excavates AI's forgotten experiments, spectacular failures and breakthrough moments, tracing how decades of wild ideas turned into the technology reshaping our world.
Episode 00 · How we got here
Listen free — one email
The episode is yours for an email address. Enter it once and it plays here and anywhere else on the site, on this browser.
Coming soon · an AI character from the show, not a real researcher.
Next · Module 01