The Thermodynamics of Intelligence, Part 1: A Magnet That Remembers

In 2024 the physics Nobel went to two AI researchers. Physicists asked how this counts. The answer is five pages long, published in 1982. Not a crossover. A claiming.

A Cheap Hotel Before Dawn

The call from Stockholm reached Geoffrey Hinton in a cheap hotel in California, before dawn on October 8, 2024. The internet connection was bad. The phone connection was worse. On the other end, the Royal Swedish Academy of Sciences was trying to tell him that he had won the Nobel Prize in Physics.

Hinton had plans that day. He was scheduled for an MRI scan. Somewhere between the phone call and the press conference that followed, he worked out that the scan would have to wait.

“I’m flabbergasted,” he told the reporters who found him within hours. He had not expected it, he said, had not even suspected. And when someone asked the obvious question, whether he considered himself a physicist or a computer scientist, he gave an answer that would fuel arguments for months: he was someone who does not really know what field he is in.

Three time zones east, at Princeton, the news reached John Hopfield. He was 91. Decades earlier, when Hopfield ran a lab at Caltech, one of his postdocs had asked him whether he expected to win a Nobel someday. Hopfield laughed the question off. There was, he pointed out, no Nobel Prize for artificial neural networks.

On the morning of October 8, 2024, there was. The Academy’s citation honored the two men jointly, awarding them eleven million Swedish kronor for foundational discoveries and inventions that enable machine learning with artificial neural networks.

The pair standing on that podium made an odd photograph. One of them spent his first twenty-five professional years calculating how light moves through crystals. The other trained as an experimental psychologist and describes his own disciplinary identity with a shrug. Between them, no telescope, no accelerator, no superconductor. The most famous prize in physics had just been handed to two men whose shared life’s work was teaching networks of artificial neurons to remember and to learn.

The physics world noticed. Then it started shouting.

Is This Physics

The complaints began within hours of the announcement, and Nature’s news desk collected them under a headline about a debate over scientific fields. Jonathan Pritchard, an astrophysicist at Imperial College London, said he was speechless, and that he struggled to find the physics discovery anywhere in the citation. Sabine Hossenfelder, a physicist with one of the largest science audiences online, filed the work where she felt it plainly belonged: computer science.

The critics had a respectable case. Hinton already held the 2018 Turing Award, computer science’s highest honor, for the very line of research the Nobel now celebrated. A single body of work collecting the top prize of two different disciplines suggested, to the skeptics, that one of the two committees had made a category error. And everyone knew which committee had moved second. The prize looked, from this angle, like the Nobel chasing the AI moment, physics dressing itself in the season’s fashion.

The defense arrived just as quickly, and it came from the top. Ellen Moons, who chairs the Nobel Committee for Physics, explained that the laureates had used tools from physics to build the methods underneath modern machine learning. Hopfield had constructed a memory that stores and rebuilds patterns. Hinton had built on it with methods drawn from statistical physics. The tools were physics; the prize honored what the tools had made.

Both sides accept something without argument. The critics say the borrowed tools do not make the work physics. The defenders say the borrowed tools do. Both camps agree that something was borrowed, that physics and neural networks are two territories with a border between them, and that the only live question is what a loan across that border is worth.

That shared premise is the actual error, and neither side touches it.

The Hopfield network is not an application of physical tools to a foreign problem. It is not physics-flavored, physics-inspired, or physics-adjacent. Written down as mathematics, it is a physical system, one that statistical mechanics had been studying under another name before Hopfield ever picked it up. The prize did not stretch the borders of physics to reach it. The borders had covered it all along.

That claim sounds like rhetoric. It is checkable. The evidence is a five-page paper, and the story of the man who wrote it explains why he was the one who could.

The Fields Moved Past Him

The Princeton physicist William Bialek once asked Hopfield a direct question: how did you decide to leave condensed matter physics for biophysics?

Hopfield rejected the question’s premise. He never changed, he said. He kept doing the same things, and “the fields moved past me.”

The record backs him up. Hopfield was born in Chicago in 1933, took his physics PhD at Cornell in 1958, and went to work at Bell Labs on the interaction of light with solid matter. His doctoral work gave condensed matter physics a piece of standard vocabulary; the field still speaks of Hopfield coefficients when it describes how photons and crystal excitations mix. From Bell Labs the career ran through Berkeley and then Princeton, a conventional and distinguished path through American physics. In 1980 he moved to Caltech, and the trajectory bent.

What pulled him was a conviction that biology’s deepest puzzles were physics puzzles wearing disguises. He would state the credo plainly in an essay decades later: “Physics is a point of view about the world.” Not a subject matter, a point of view. A physicist, on this view, is someone who looks at any system, wet or dry, living or dead, and asks how its collective behavior follows from the interactions of its parts. Most of his colleagues took the subject-matter view instead, and by that standard a physicist writing about neurons had wandered off the reservation.

The institutional record of that wandering survives in the header of a single article. In April 1982, the Proceedings of the National Academy of Sciences published five pages titled Neural networks and physical systems with emergent collective computational abilities. Sole author: J. J. Hopfield. Affiliations: the Division of Chemistry and Biology at Caltech, and Bell Laboratories.

Read that header slowly. A career physicist, publishing the work of his life, and the letterhead says chemistry and biology. Physics departments in 1982 had no shelf for this. The paper about neural networks did not appear in a physics journal, and its author did not sign it as a physicist, because the discipline’s official map showed nothing at those coordinates.

Forty-two years later, the same discipline gave that paper its highest prize. Hopfield’s sentence to Bialek is the entire story in nine words. He stood still. The map moved.

What was on those five pages that eventually forced the map to move? A magnet. A very old, very simple model of a magnet, wearing a new name.

Spins Become Neurons

The model was born in 1920, in Hamburg, as a homework problem.

The physicist Wilhelm Lenz handed his doctoral student Ernst Ising a stripped-down picture of magnetism. Take a row of atoms. Give each one a tiny magnetic arrow that can point only up or only down. Let each arrow feel only its immediate neighbors, with an interaction that rewards alignment. Two neighbors pointing the same way lower the system’s energy; two neighbors disagreeing raise it. The question Lenz posed: does this caricature contain the essential miracle of a magnet, the moment when trillions of arrows spontaneously agree on a direction?

Ising ground through the one-dimensional mathematics and published the answer in 1925. The answer was no. A single chain of arrows never manages collective agreement; thermal jostling always wins. Disappointed, Ising concluded the model was a dead end and left research not long after. He was wrong about the dead end in the most spectacular way available to a scientist. In 1944 Lars Onsager solved the two-dimensional version exactly and showed that on a flat grid the arrows do agree, all at once, below a sharp critical temperature. The caricature contained the miracle after all. In the decades that followed, the Ising model became the fruit fly of statistical mechanics: the minimal organism in which every theory of collective behavior and phase transition gets tested first.

Hold the model’s anatomy in mind, because you are about to see it again. Binary units. Pairwise couplings that say agree or disagree. A total energy summed over all the pairs. Dynamics that slide the system downhill toward lower energy.

Now, 1982. Hopfield’s five pages perform a substitution so simple it fits in a table. An arrow pointing up or down becomes a neuron firing or silent. The coupling between two arrows becomes the synaptic weight between two neurons, its strength set by experience. The energy function is carried over without modification.

Without modification. That phrase is why the borrowing debate of 2024 misses the point. A metaphor sets two similar things side by side. Hopfield did not do that. He wrote one mathematical object and gave its variables two readings. The magnet and the memory share an energy function the way a person shares a skeleton with their own X-ray. There is nothing to translate, because nothing differs.

One complication makes the story richer. In Hopfield’s version the couplings are not uniformly friendly. Some pairs of neurons want to agree, others want to disagree, and the demands conflict: a neuron can find itself pulled both ways by different partners. Physics had a name for magnetic systems with exactly this kind of frustrated, conflicting wiring: spin glasses. Frustration wrecks the shape of a system’s energy landscape, in a way that turns out to be useful. Instead of one smooth basin with a single bottom, the landscape wrinkles into a terrain of many separate valleys, each one a locally stable state, each one a different configuration the system can settle into and stay.

To physicists, that ruggedness was mostly a source of headaches. Hopfield looked at the wrinkled landscape and saw storage.

His construction works like this. Choose the patterns you wish to remember, say a set of images, each one a specific configuration of firing and silent neurons. Then set the synaptic weights by a simple rule, so that each chosen pattern sits precisely at the bottom of its own valley. The memories are not filed anywhere. They are terrain. Recall is what gravity does: hand the network a corrupted fragment of some stored pattern, which is to say, place the system partway up a hillside near one of its valleys, and let the dynamics roll it downhill. It comes to rest at the bottom, and the bottom is the complete, clean, original pattern.

Engineers had a name for the ability to summon a whole memory from a piece of it: content-addressable memory. They had chased it for years as a hard design problem. Hopfield’s table revealed it as a free gift of statistical mechanics, sitting in the textbooks all along, filed under relaxation to equilibrium.

The paper’s title had announced the thesis in advance, for anyone reading carefully: neural networks and physical systems, with emergent collective abilities. Not networks described by analogy with physical systems. Networks and physical systems, one category, exhibiting together the kind of collective behavior that is statistical mechanics’ entire subject.

Physics could have shrugged. What happened instead, three years later and seven thousand miles away, is the reason the 2024 prize should surprise nobody.

The First Adoption

In Jerusalem, at the Hebrew University, three physicists read Hopfield’s paper: Daniel Amit, Hanoch Gutfreund, and Haim Sompolinsky. All three worked in statistical mechanics, and they drew the conclusion the paper’s mathematics demanded. If this network is a spin glass, then it is not a subject for admiration from across a disciplinary fence. It is a system, and systems get solved.

So they solved it. They brought out the heaviest machinery their field owned, the apparatus built over decades to crack disordered magnets, and in 1985 they published the results in physics’ core journals. They mapped the Hopfield network’s phase diagram as if it were any other material: here the phase where the system behaves as a working memory, here the phase where thermal noise melts recall away, here a glassy phase where the landscape holds only meaningless valleys. A model of the brain, charted like a magnet.

Out of that calculation came a number, and the number became famous: 0.138.

Here is what it measures. Keep loading patterns into a network of N neurons and the energy landscape gets crowded; valleys press against valleys. The Jerusalem trio proved that the loading ratio, patterns stored per neuron, has a sharp ceiling near 0.138. Below the ceiling, the memory works: every stored pattern owns a genuine valley, and recall finds it reliably. At the ceiling, the whole structure fails, and it fails in a manner worth staring at. The network does not get gradually forgetful, blurring a memory here and there. Every valley destabilizes together. One pattern past the limit and retrieval collapses across the board, from nearly everything to nothing, the way a flooded valley does not half-drown.

Statistical mechanics has a word for behavior of this shape, a smooth dial crossing a threshold and the system’s character breaking all at once. It is a phase transition. Water at zero degrees, iron at the Curie point, a Hopfield network at α = 0.138: the same mathematics of rupture, computed with the same tools, and in this case printed in Physical Review.

As an institutional act, the 1985 papers were something particular. Three physicists took a neural network model, subjected it to their field’s most rigorous machinery, and published the results in their field’s own journals, without ever framing the exercise as interdisciplinary outreach. From where they stood, no border had been crossed, because the object in front of them was a frustrated magnet that happened to have been wired up as a brain. Their phase diagram was a property deed. Statistical mechanics had examined the Hopfield network and recorded it, formally, as one of its own.

The Nobel committee’s claiming in 2024 was thirty-nine years late to a claiming the field itself had already performed on paper.

Three Questions

The 1985 calculation does one more thing for us: it hands this series its method.

Every article in The Thermodynamics of Intelligence will take one pairing of a physics concept and an AI phenomenon and put it through the same interrogation. Three questions, no substitutions accepted.

What is the order parameter? Name the quantity whose jump certifies that a qualitative change has occurred, not a metric that improves smoothly, but one that marks a before and an after.

What is the control parameter? Name the external dial, whatever an experimenter or an engineer actually turns.

Where is the critical point? Point to the value of the dial at which the jump happens, in a specific paper, in a specific figure, in data someone can check.

Run the Hopfield network through the test. The order parameter is the memory overlap, written m: the match between the network’s current state and a stored pattern. Successful recall is m leaping from near zero to near one; the leap, not a climb, is what makes recall an event. The control parameters are two: the storage load α and the temperature T, the noise level in the neurons’ updates. The critical point, for a noiseless network, is α_c ≈ 0.138.

None of these answers required interpretation on my part, and that is the point of starting the series here. The Jerusalem physicists wrote the phrase order parameter themselves, in 1985, doing ordinary physics. The three questions are not a lens this series holds up to the literature. They are the literature’s own native grammar, and the wager of the next eight articles is that the grammar keeps working: on the annealing schedule inside a Boltzmann machine, on the sudden generalization that researchers call grokking, on the plateaus where training stalls, on the fight over whether large models’ emergent abilities are real or a trick of measurement. Where the questions produce answers anchored in data, we will have found a genuine correspondence. Where they produce nothing, we will say so, because a test that cannot fail is not a test.

The Claiming

October 8, 2024, one last time.

Princeton convened a press conference for its 91-year-old laureate, and someone asked Hopfield about the systems his work had led to, the vast modern networks and their unnerving capabilities. His answer reached back over his whole career. When a system becomes rich enough in size and complexity, he said, it shows properties you could never infer from the elementary parts you put into it. And then the sentence that settles the week’s argument: you have to say such a system “contains some new physics.”

Set that against the debate that had raged since the announcement. The defenders of the prize had argued that Hopfield and Hinton borrowed physics’ tools and deserved credit for the borrowing. The critics had argued that borrowed tools do not transfer citizenship. Hopfield, given the microphone, declined both positions. He said the collective behavior of many simple units is physics, is what physics is about, whether the units are spins in iron or neurons in a network or weights in a machine no one fully understands. It is what his paper’s title had asserted in 1982, when no physics journal had a place for it. He had been saying one sentence for forty-two years, and the discipline had finally arrived within earshot.

His other sentence, the one he gave Bialek, now reads as prophecy in reverse. The fields moved past him in the 1970s, when he carried a physicist’s point of view into territory the subject-matter map did not cover. They moved again in 1985, when Jerusalem drew the phase diagram. They moved again through the decades in which statistical physicists quietly colonized learning theory. And they moved conclusively on an October morning in 2024, when the Royal Swedish Academy of Sciences updated the official map to show what the unofficial one had shown for decades: that the border of physics runs on the far side of the neural network, and always did.

Hopfield never moved at all. He arrived early, by about forty years, and waited for his field at the destination.

What Thermodynamics Says

Order parameter: the memory overlap m. Recall is a jump in m, not a climb.

Control parameters: the storage load α and the temperature T.

Critical point: α_c ≈ 0.138, computed in Jerusalem in 1985, thirty-nine years before the Nobel.

---

The Thermodynamics of Intelligence. When statistical mechanics met neural networks: a forty-year story of a discipline claiming its own.