This is the overture to The Attention Wager, a close reading of Attention Is All You Need, one section and one bet at a time.
In June 2017, eight researchers at Google submitted a paper with a title that sounded more like a slogan than a scientific claim: Attention Is All You Need. The claim itself was a bet, and a radical one. Every serious sequence model up to that point processed language recurrently, one word feeding into the next, because that seemed like the only way to represent order. The paper’s authors proposed throwing recurrence away entirely and keeping only one mechanism, borrowed from a much older field than machine learning: the psychology of attention, the same faculty William James described a century earlier as the mind choosing, out of many possible objects, which one to attend to.
The bet paid off, eventually, spectacularly, and not in the way anyone predicted at the time. This series is a close reading of that paper, section by section, seven years and one AI revolution later. We return to it knowing how the story unfolded, looking for what the first read missed, and for what the room that wrote it could not yet have known it was starting.
Each part anchors to one piece of the paper, its own section, its own equation, its own paragraph, and asks the same question of it: what did this specific choice bet, what did the field’s next seven years pay out, and is the position closed or still open. Multi-head attention got one paragraph of justification in 2017. It took the field years afterward to explain why it worked. Positional encoding was a near coin flip between two options that tested almost identically. Nine years later, the coin is still in the air.
Eight parts, eight bets, one paper that turned out to be righter, and more contested, than anyone in that room could have known.
Every part ends the same way, in a section simply called The Bet: what was staked, what the field’s next seven years paid back, and whether the position is closed or still open. Some of these wagers are unambiguous wins, still standing exactly as proposed. Others did not survive contact with the following decade, and this series says so plainly rather than defending a paper it also, clearly, admires.
Eight Bets
Eight parts, in the order the paper argues, all filed under The Attention Wager:
Part 1: The Opening Bet: why walking away from recurrence looked reckless in 2017, and the psychology-textbook detour hiding at the start of a machine-learning paper.
Part 2: Three Matrices: the three-projection anatomy of attention, worked out once and never re-derived anywhere else in this series.
Part 3: Many Heads: one paragraph of justification for a design choice that took the field five years to explain.
Part 4: Where Order Comes From: a coin flip the authors admitted was a coin flip, and the nine-year-old bet it left still unsettled.
Part 5: Two Thirds: the unnamed component holding two-thirds of the model’s parameters, and the three sentences the paper spent on it.
Part 6: The Fork: how one symmetric architecture split into three lineages, and why the least symmetric one is the one still standing.
Part 7: Judgment Day: the efficiency argument’s own fine print, and why it now reads like the strongest case for the other side.
Part 8: Closing the Book: eight authors, seven years, and the one bet in this whole series that never got settled.
If This Is You
If you want to be able to compute attention by hand, not just gesture at the diagram, Part 2: Three Matrices is built to leave you able to. Nothing later in the series re-derives the three projections, so this is the part to read slowly, with the paper open beside it.
If multi-head attention has always felt like an engineering hunch that got lucky, Part 3: Many Heads takes the one paragraph the original paper spent justifying it and follows what it took the field years to prove.
Maybe positional encoding is the part of the Transformer you have never fully trusted. Part 4: Where Order Comes From is this series’ single most contested bet, tested against a coin flip in 2017 and still open nine years on.
If you are reading for where the money and the chips are, Part 7: Judgment Day is where the efficiency argument the paper made in 2017 turns into the strongest argument against it.
And if you want the whole story rather than one mechanism, start at Part 1: The Opening Bet and read straight to Part 8: Closing the Book, where every wager placed across the series gets settled, all eight authors’ later paths get traced, and the question this series opened with, whether machine attention and human attention are the same thing, finally gets answered.
The paper’s title was already a wager. This series has spent eight parts collecting, section by section, on the terms the authors themselves set.