Mathematical Thinking for AI, Lesson 1: Linearity
Ten times the input, ten times the output, no surprises. That refusal of surprise is why anything computes at all, and it quietly forbids the one thing intelligence needs most.
This is Mathematical Thinking for AI, a course in the twelve ideas underneath modern machine learning, and what each one costs. No formulas.
The picture
Linearity is a world without surprises.
Do two things together and the result is exactly what you would have gotten doing them separately, then adding. Scale the input by ten and the output scales by ten, no more, no less. Nothing interacts. Nothing emerges. Add the causes and you have added the effects, forever, in every direction, at every scale.
The world is not like this. Stretch a spring twice as far and, soon enough, it gives rather than pulling twice as hard. Double a drug dose and you do not double the effect. Put two people in a room and what happens is not the sum of what each would do alone.
We know all this. And we insist on pretending otherwise, everywhere, all the time, because a world without surprises is the only world we can compute in.
And “compute” here means something specific and humble. In a linear world, you can understand a system by interrogating it one simple input at a time. Feed it pure cases, record what comes back, and you are done: any complicated input is a mixture of your pure cases, so its output is the same mixture of your recorded answers. Divide, probe, recombine. Whole fields of engineering that speak of “response” lean on this, and so does every unit test you have ever written against a system you hoped was modular. The hope that testing parts tells you about wholes is the hope of linearity, wearing work clothes.
In one sentence: in a linear world, the whole is exactly the sum of its parts, no more, no less, and nothing appears in the combining. That is the whole gift, and it is also the whole problem.
