The Thermodynamics of Intelligence, Part 5: How to Miss a Phase Transition

The first four critical points here were calculated. This one was seen, in a curve nobody had reason to keep watching. Then the arguing began over whether it was there at all.

Seen, Not Derived

Every critical point this series has produced so far came off a blackboard. Amit, Gutfreund and Sompolinsky calculated 0.138 in 1985 with the machinery of disordered magnets. The boundary where memory melts was drawn on a phase diagram in 1987. Bruce Hajek proved the exact cooling threshold in 1988, and no computer was required to find any of them.

This one is different. It was noticed in 2022, on a training curve, by people who had left a small model running long past the point where anything was supposed to happen. Nothing was predicted. Something showed up.

And then the interesting part. Within a year, one group had argued that the jump everyone could see was not really a jump, and that the change had been happening smoothly all along beneath a metric too crude to show it. Within two years, another group had argued that it was a phase transition after all, of a specific and classical kind, and had produced the hysteresis to prove it.

Both were right, and the reason they were both right is the most useful thing in this article. A phase transition is not something you look at. It is something you measure, and measuring it requires knowing which quantity to put on the vertical axis. Choose wrong and a real transition becomes invisible, or an artefact becomes a discovery.

Nobody Trains This Long

The setup is almost aggressively small.

Everyone already knows one case of the task, from clocks. Four hours after ten o’clock is two o’clock, not fourteen o’clock. Counting climbs to twelve and then starts over at one, so that no matter how large the numbers going in, the answer always lands somewhere on the same twelve-hour face.

Now run that with 113 in place of 12, with one adjustment. A clock face starts its count at one and ends at twelve; this arithmetic starts at zero, so the numbers run from zero to 112, which is still 113 of them. Add two of them, and if the total reaches 113 or more, subtract 113 and keep what is left, so every possible answer falls back inside that same range. Mathematicians call this modular addition, and the number you count up to before starting over, 113 here, is the modulus. A prime is chosen for it because a prime leaves no smaller cycles hidden inside the big one, which keeps the structure the network has to find as simple as such a structure can be.

The task makes a good laboratory because the whole of it is finite and known in advance. There are 113 possible first numbers and 113 possible second numbers, so there are 12,769 questions in existence, and a table containing all of them is complete knowledge of the subject. Nothing is unknown, nothing is noisy, and no more data could ever be collected.

So: build that table, hide most of it, show a small transformer what remains, and ask it to produce the entries it has never seen. There is nothing here of the scale that makes modern machine learning expensive. The entire problem would fit on a few sheets of paper.

Now train it, and watch two numbers. Training accuracy, meaning performance on the examples the model has seen. Test accuracy, meaning performance on the ones it has not.

Training accuracy climbs fast and hits one hundred percent. The model has memorized its examples, and by every convention of the field this is where the run ends. The model has fit its data, the test accuracy is stuck at chance, and what you have is a textbook case of overfitting: a network that memorized rather than learned.

So you stop. Any competent practitioner stops, and stops for good reasons rather than out of impatience. The training loss is at zero and cannot go lower. The validation curve has been flat for longer than anything in your experience suggests is meaningful. Compute costs money. And the standard technique for exactly this situation, early stopping, exists to formalize the judgment that further training past this point does harm rather than good. You note that the dataset was too small, you write it up as a negative result, and you move on to the next experiment.

That practitioner is the protagonist of this article, and everything below is a description of what they missed.

Test accuracy, meanwhile, sits at the floor. Not creeping. Flat. It stays flat while training accuracy remains pinned at perfect, and it stays flat for a very long time, through thousands more passes over data the model has already completely absorbed, with the loss on those examples already at zero and no visible reason for anything to change.

Then the test accuracy climbs off the floor and goes to nearly perfect.

The word for what has to happen for anyone to see this is patience, but that is not quite it either, because patience implies expectation. Nobody keeps training a model that has already memorized its training set and shows no sign of generalizing, for the same reason nobody keeps stirring a pot after the recipe says stop. The behavior was available to be discovered at any point in the previous decade. It required an experiment that no reasonable person had reason to run.