What if I told you that everything we call “AI” is just one operation, repeated trillions of times?
16 min read
1 day ago
--
TL;DR
ChatGPT doesn’t think. It multiplies matrices.
Stable Diffusion doesn’t imagine. It multiplies matrices.
AlphaGo didn’t strategize. It multiplied matrices.
Every breakthrough in AI for the past decade reduces to the same primitive operation: matrix multiplication. Specifically, variations of one elegant formula:
Attention(Q, K, V) = softmax(QK^T / √d) V
Three matrix multiplications. One softmax. That’s it.
And somehow, when you do this operation at sufficient scale, with enough parameters, across enough layers, something impossible happens.
The system learns to write poetry. To prove theorems. To generate photorealistic images of things that never existed. To reason about counterfactuals. To translate between languages it was never explicitly taught.
This essay is about why matrix multiplication, the most boring operation in linear algebra, becomes the most profound operation in intelligence when performed at scale.
What Awaits You
- The Illusion of Complexity: Why AI looks harder than it is
- Matrix Multiplication…