The Attention Wager, Part 8: Closing the Book
Seven of eight on a stage together, seven years late. Eight bets settled. A paper named after one third of itself, by people right more often than they could explain.
This is The Attention Wager, a close reading of Attention Is All You Need, one section and one bet at a time.
On Wednesday the twentieth of March 2024, in a conference room in San Jose, seven people sat in a row on a stage.
Left to right: Łukasz Kaiser, Noam Shazeer, Aidan Gomez, then Jensen Huang in the middle as host, then Llion Jones, Jakob Uszkoreit, Ashish Vaswani, Illia Polosukhin. Three on one side, four on the other. NVIDIA’s conference ran more than nine hundred sessions that week and this was the fullest room at any of them.
Two things about that row are worth holding onto.
The first is that it was seven and not eight. Niki Parmar was not there. NVIDIA’s announcement a month earlier had promised a panel with all eight authors of the paper, and on the day it had seven chairs filled and one name missing from the row.
The second is that it was the first time.
Seven years after the paper appeared, and after it had been cited past a hundred thousand times, and after an industry had been built on it, this was the first occasion on which the people who wrote it had appeared together in public. Not the first since they scattered. The first, full stop. They had worked in the same building in 2017 and had not been in the same room since, and it took a hardware company’s annual conference to arrange it.
“Everything that we’re enjoying today can be traced back to that moment,” Huang told the room. Then, having filled the largest space at his own conference with people who had come to see them, he sat down in the middle of the row and asked them what came next.
By then none of them worked at Google.
