Robonaissance

Robonaissance

The Attention Wager

The Attention Wager: A Close Reading of Attention Is All You Need
Attention Is All You Need was a bet. This series collects on it, section by section. Seven wagers closed. One is still open. Eight parts, one argument.
 • Hugo
The Attention Wager, Part 8: Closing the Book
Seven of eight on a stage together, seven years late. Eight bets settled. A paper named after one third of itself, by people right more often than they…
 • Hugo
The Attention Wager, Part 7: Judgment Day
The efficiency argument had a condition printed beside it: that sequences stay shorter than the model is wide. Attention was borrowed from a theory…
 • Hugo
The Attention Wager, Part 6: The Fork
Two stacks, drawn symmetrically, for a task with an input and an output. The field cut the figure in half twice. The oldest diagram of language in the…
 • Hugo
The Attention Wager, Part 5: Two Thirds
Three sentences for two thirds of the parameters. The component with no name turned out to hold what the model knows. Memory that is not attention, and…
 • Hugo
The Attention Wager, Part 4: Where Order Comes From
Two options, tested, tied. The tie was broken by a prediction that failed. Nine years later the question is still open, and the oldest version of it was…
 • Hugo
The Attention Wager, Part 3: Many Heads
Eight heads, one sentence of justification, and five years before anyone could say why it worked. The mechanism that binds separate features into a…
 • Hugo
The Attention Wager, Part 2: Three Matrices
Three matrices, one softmax, and a denominator nobody quotes. How a lookup table learned to blend. The oldest idea in memory, finally made…
 • Hugo
The Attention Wager, Part 1: The Opening Bet
A bottleneck the whole field had agreed to live with. Eight researchers bet it could go entirely. The mechanism they reached for was already a century…
 • Hugo
© 2026 Robonaissance · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture