The Thermodynamics of Intelligence, Part 6: An Exact Mapping, Approximately
Physics owns a machine for deciding which scale to look at. In 2014 two physicists said deep networks run the same one, and put the word exact in the title. It is still contested.
A Machine for Standing Back
Part 5 ended on a problem rather than an answer. A transition can be real and invisible at once, depending on which quantity you plot, and picking the quantity that makes it visible is not a preliminary to the physics. It is the physics, and it is the part that took a century to learn.
Physics eventually built machinery for it. The machinery describes how a system’s description changes as you step back and look at it more coarsely, which features persist through the coarsening and which vanish, and it won a Nobel Prize in 1982 for explaining why materials with nothing in common undergo identical transitions.
A deep network has layers. Data enters at the bottom, each layer produces a compressed description of what the layer below handed it, and features that survive to the top are the ones the network decided mattered. Stated that way the two procedures sound like the same procedure.
In October 2014 two physicists said they were, and did not hedge. Their title contains the word exact.
What happened next is the reason this instalment exists. The claim was attacked with a counterexample, defended in a published comment, tested by other groups against the simplest system anyone could try it on, and left standing in a condition that is neither vindication nor refutation. Every correspondence in this series so far has been a literal identity: the same Hamiltonian, the same correlation functions, the same phase diagram. This one goes as far as resemblance and stops, and the place it stops can be located precisely.
