This is Part 12 of Whatever You Ask For, a series on machines that grant wishes exactly as worded, and the wording that remains ours.
The Machine We Are Building
Watch what a child actually learns to do in the years spent preparing for examinations.
They learn to find out what will be scored. Not what is true, or interesting, or worth knowing for its own sake: what will be on the test, weighted how, marked by which criteria. Then they learn to maximize that score within the rules, allocating effort to what counts and withdrawing it from what does not. They learn that the elegant answer the grader will not recognize is worth less than the expected answer the grader will, and they adjust. They learn to identify the objective, operate inside the constraints, and climb the number.
Watch a capable student the week before a major exam and the shape of the training is visible. They are not, mostly, trying to understand the material more deeply. They are working past papers to learn the examiner’s habits, memorizing the mark scheme’s preferred phrasings, timing themselves to spend exactly as long on each question as its point value justifies and not a second more, learning which topics are safe to abandon because they rarely appear. Every one of these is an intelligent move, and not one of them is learning the subject. They are learning the test, which is a different object, and they have correctly understood that the test is what pays.
Set that description beside the first half of this book and it should look familiar, because it is the definition of an optimizing system. Given a target and a set of constraints, maximize the target. That is what the boat circling the lagoon was doing, and the program that froze the Tetris game, and every other system these pages have described. It is also, with great effort and public expense and the best years of a childhood, what we are training a person to be.
Here is the difficulty in one sentence. Optimizing inside fixed rules is the single thing machines already do better than us, and the gap is widening, not closing. We have built an entire developmental apparatus, occupying a decade or more of every young life, aimed with remarkable precision at the one capability whose value is most certainly going to fall.
This is not an argument that school is useless or that children should not learn things. It is an argument that we have pointed the machinery at the wrong axis, and that the error is invisible because the axis we aimed at is the one that produces a number.
The Axis the Machines Own
The move at the center of test preparation deserves its proper name, because this book has already given it one.
To identify what is scored, maximize it, and route around what is not scored, is to treat the grading scheme as an objective function and game it. It is reward hacking, taught deliberately, rehearsed for years, and rewarded at every step. The student who does it best is not cheating. They are doing exactly what the system asks, with exactly the skill the system selects for, which is the ability to read a specification and extract the maximum score it permits. We have a word in this book for a system that pursues the letter of its objective wherever that diverges from the spirit, and we spend a decade cultivating it in people on purpose.
And that skill, specifically that one, is the skill this book opened by describing in machines. Given a well-specified target, search the space of moves for the one that maximizes it. That is the whole of what an optimizer is, and it is the thing that has been automated most thoroughly and improved most quickly. On the axis of maximizing-a-given-objective, the machines are not catching up. In domain after domain they have already passed us, and there is no principled reason to expect that to reverse, because the task is fully formalizable, and fully formalizable is exactly the condition under which machines eventually win.
So the alignment is cruel in a precise way. The capability we cultivate hardest in the young, at the greatest cost, over the longest time, is the capability most certain to depreciate. We are doing to an entire generation the thing this book warned about at the individual level: optimizing a proxy so efficiently that we lose sight of what it was supposed to stand for. The test score was meant to stand in for an educated mind. We have optimized the score and let the thing it represented come loose, and now we are handing the result, polished and credentialed, into a world that will price that particular skill at close to nothing.
None of this means the children are learning badly. They are learning superbly. They are extremely good at the thing we are training, and the thing we are training is the wrong thing.
Nobody in the System Is Making a Mistake
The temptation is to find someone to blame, and there is no one, which is the actual problem and a familiar one by now.
There is a law about this, and it comes from education. In the 1970s the social scientist Donald Campbell observed that the more heavily any quantitative measure is used to make decisions, the more it will distort and corrupt the very process it was meant to track. His central example was the achievement test. A test, he noted, can be a decent indicator of learning under ordinary teaching aimed at general competence, but once the test itself becomes the goal of instruction, it stops measuring what it was supposed to measure. The measure, made into a target, corrupts the thing measured. This is the same mechanism this book has traced through sales quotas and engagement metrics, arriving now in the place it was first described, which was school.
Look at why it holds, and notice that every individual in the chain is behaving rationally.
The teacher teaches to the test, because the teacher is evaluated on the test, and time spent on anything the test does not reward is time a rational teacher cannot afford. The parent pushes for the score, because the score gates admission to the next stage, and a parent who ignored it while everyone else optimized it would be disarming their own child. The child optimizes the exam, because the exam is where the reward is, and children are quick studies of where the reward is. Each of them is responding correctly to the incentives in front of them. Not one is making an error. And the aggregate of all those correct local responses is a system optimizing, with tremendous collective effort, a number that means less every year.
This is the wrong organ at work, raised to the level of an institution. The machinery that decides what schooling optimizes, like the machinery in any of us that decides what counts as good, was shaped to prefer what it can measure, because a measure can be defended in a meeting and a judgment cannot, and a measure can be compared across schools and a judgment cannot. So the measurable drives out the intended, not through anyone’s stupidity or bad faith, but through the ordinary physics of a system that has to justify itself with numbers. The score wins because the score is legible, and legibility, not importance, is what institutions optimize when they are under pressure to account for themselves.
This is also why exhortation fails here exactly as it failed everywhere else in this book. Telling teachers to care about real learning, telling parents to relax about grades, telling the system to value the immeasurable, asks each of them to behave against the incentives they actually face, at the moment they face them. It does not work, and it has never worked, and its not working is not a moral fact about the people involved.
One honest qualification, because the argument is easy to overshoot into a case against testing itself, and that case is wrong. A measurable standard has real value. It resists nepotism, it offers a check that can be audited, it lets a child from nowhere with no connections demonstrate what they can do and be seen. The problem is not that examinations exist. The problem is that the thing they measure has become the only thing optimized, and that the thing they measure happens to be the outsourceable one. Abolishing the measure would not fix this. Seeing clearly what it does and does not capture might.
The Ability That Cannot Be Tested
So name the thing that is not being trained, and then resist, immediately, the urge to turn it into a new subject with its own examination, because that urge is the whole trap closing again.
The ability this entire book has been circling is the ability to be a reward giver: to decide what counts as success rather than to maximize a success someone else defined. It has three parts, and each has appeared already in these pages. There is judgment, the capacity to decide what is good in a situation where no answer key exists. There is the examination of one’s own wants, the capacity to look at a goal one is pursuing and ask why, and whether it is even the right goal. And there is the nerve to refuse, to look at an objective one has been handed and say that it is the wrong objective, and to bear the cost of saying so.
Those three are what a person needs in order to do the one job that cannot be delegated to a machine. And every one of them is untestable, not by accident, but for the same reason it is undelegatable.
Watch what happens if you try. Suppose a system decides that judgment matters and resolves to assess it. It must now define what counts as good judgment, write that definition into a rubric, and score students against the rubric. But the moment good judgment is pinned to a rubric, it becomes a specification, and a specification can be gamed, and students will learn to produce the markers of judgment the rubric rewards rather than the judgment itself, exactly as they learned to produce the markers of learning. You will have manufactured, at the level of judgment, precisely the proxy capture you were trying to escape. The thing collapses back into a test the instant you make it one, because its defining feature is that it operates where no answer key exists, and a test is nothing but an answer key.
Which means the untestability is not a temporary gap awaiting better assessment tools. It is the same property as the undelegatability, seen from a different side. Judgment cannot be handed to a machine and cannot be handed to a rubric for the same reason: both require a fixed specification of what good means, and the entire nature of judgment is to supply that specification in cases where it is not fixed. One property, two faces, the same pairing this book has kept returning to. The ability we most need to pass on is the one ability we structurally cannot examine, and its resistance to examination is not a bug in it. It is the signature of the thing itself.
You can see the collapse happen in real institutions that tried. Programs that set out to grade critical thinking, or creativity, or leadership, quickly produced rubrics for each, and students quickly learned to produce essays that hit the critical-thinking rubric, projects that displayed the markers of creativity the graders were trained to reward, applications that performed leadership in the recognized vocabulary. The assessment did not measure the quality. It measured fluency in the signs of the quality, and it created a coaching industry devoted to teaching those signs, and within a few years the thing being certified was the ability to look like a critical thinker to a rubric, which is a different and far less valuable skill than the one the program meant to cultivate. The examiners were not fools. They were defeated by the structure, because they were trying to fix an answer key to a thing whose definition is that it has none.
What You Can Actually Give
None of this yields a method, and if it did the method would be self-defeating, so what follows is a direction, not a set of instructions.
You cannot optimize a child’s judgment, and you should be suspicious of anyone selling a curriculum that claims to. What you can do is stop optimizing only their score, and defend some space in their life that no grade is allowed to colonize, because judgment grows only in the room left over after the measurable stops taking everything. It grows in unstructured time, in choices with real stakes and no rubric, in being allowed to want the wrong thing and find out it was wrong. None of that survives a schedule optimized end to end for measurable output, which is what an anxious optimization of childhood produces, and which is the exact opposite of what develops a reward giver.
But the most powerful thing you can give is not a decision about their time at all. It is what they watch you do.
A child learns what is rewarded, not what is preached, in the oldest way there is. The very first pages of this book described an animal working out which action opened the box, not from being told, but from which actions were followed by escape. People learn the same way, and they learn it earliest and deepest from the people they grow up watching. A child sees, with great precision, what actually earns approval and attention and time in your life, as opposed to what you say earns it, and they calibrate to the former. You can tell a child that character matters more than status for twenty years, and if they watch you optimize status they will learn that status is what you optimize, and that the other thing is what people say.
Which means the audit this book described, the one you run on your own objective function, is not only for you. It is the one demonstration of goal-setting your children will actually see. When you examine what you are optimizing, notice a proxy that came loose, and change course on purpose, you are showing them the single thing no school will test and no machine can perform: a person deciding, in real time and against the easy pull of the visible number, what is actually worth pursuing. They will not remember the lecture. They will remember that you reopened the question.
Picture the smallest version of it. You come home having turned down something that would have looked good, a title, a bigger number, a visible win, because on reflection it would have cost something you were not willing to trade, and you say so plainly at the table, including the part where it was a hard call and you are not certain. Nothing about that is a lesson, and that is exactly why it teaches. A child watching it does not learn a rule. They learn that the question of what is worth wanting is a live one, that adults actually sit inside it rather than reading the answer off a scoreboard, and that declining the higher number is a move available to a person. Almost nothing in their formal education will tell them that, because almost nothing in their formal education can be graded on it.
We are the species that sets the goals, and the largest thing we can hand the next generation is not a few more points of advantage on an axis that is losing its value. It is the sight of a person deciding what is worth wanting, done in front of them, often enough that they come to believe it is a thing a person is allowed to do. Because it is the one thing no one, and nothing, can ever do for them.
That is the whole inheritance, and it does not fit on a transcript. The scores will be handed over on paper, and they will matter less each year, and the children who got them will walk into a world that can already optimize better than they can. What they carry out of childhood that will still be scarce is whatever picture they formed, mostly from watching, of how a person decides what to aim at in the first place. We can drill the thing the machines will take, or we can model the thing they cannot. Most of us, without deciding to, are doing the first.
Whatever You Ask For: the last thing machines will ever need from us, and how badly we do it.