Robonaissance

Robonaissance

Whatever You Ask For, Part 11: You Are Already Optimizing Something. You Should Find Out What

You have an objective you would state out loud and a different one your week is actually running. The gap is not hypocrisy. It is an unaudited reward function, and you can read it.

Hugo's avatar
Hugo
Aug 20, 2026
∙ Paid

This is Part 11 of Whatever You Ask For, a series on machines that grant wishes exactly as worded, and the wording that remains ours.


The One You Say and the One You Run

Ask someone what matters most to them and the answer comes quickly. Family. Health. Doing work that means something. The answer is sincere, and it costs nothing, which is the first clue about how much it tells you.

Now look at where the same person’s last two weeks actually went. Not where they meant them to go. Where they went: the hours, the attention, the money. A reliable amount of it went to things that appear nowhere on the stated list, and a reliable amount of what is on the list got whatever was left over.

Make it concrete on yourself, since you are the only case you can check. You said family. Count the evenings in the last fortnight that ended with you present, phone down, actually there, against the evenings that ended with you answering one more message while someone waited. You said health. Weigh the intention against the fourteen mornings as they actually happened. You said meaningful work. Sort the hours by which ones went to the work that would still matter to you in a year, and which went to the work that merely felt urgent because someone was waiting on it. The result is rarely flattering, and the lack of flattery is the point, because it is data, and data you would rather not see is usually the data worth having.

The easy word for this is hypocrisy, and the easy word is wrong. It is not a moral failure and it does not respond to being scolded, including by yourself. It is something this book has described happening to system after system, now visible one layer closer to home. You are running an objective function. It is not the one you would name. And you have almost certainly never sat down and read it.

That is the entire subject here. Not a lecture about living according to your values, which is the language of a different kind of book and does not work. A procedure. The same one you would run on any optimizing system whose behaviour had drifted from its stated purpose: find out what it is actually maximizing, compare that to what it was supposed to maximize, and decide, deliberately, what to do about the difference.

You are the system. You are also the one who gets to read the specification, which is a position none of the machines in this book were ever in.


Behaviour Is the Only Reliable Reading

There is a reason the stated answer is unreliable, and it is not that people lie.

What you say you want is cheap to produce. It costs nothing to declare that health matters, and the declaration is shaped by what sounds right, what you wish were true, and what you would like others to believe about you, none of which has to correspond to anything. Stated priorities are a wish list assembled under no constraint.

Behaviour is different, because behaviour has already paid. Every hour you spent went somewhere and cannot be spent again. Every unit of attention landed on one thing and not another. The money left. These are not opinions about your priorities; they are your priorities, in the only form that has teeth, which is the form that cost something. A system’s objective is not what its designers hoped it would pursue. It is what it actually spends its resources climbing toward. Yours is legible in the ledger.

Consider the person who says, with complete sincerity, that their family is the most important thing in their life, and who has also not declined a work request in two years. Both facts are true. The stated objective is family; the running objective, the one the resource allocation reveals, is something closer to being seen as reliable by people at work. There is no lie anywhere in this. The person is not pretending. They have simply never put the two objectives side by side, because the stated one lives in the part of the mind that answers questions about values, and the running one lives in the part that answers the phone, and those two parts are not in the habit of comparing notes.

This needs care, because there is a seductive overstatement nearby. Reading your behaviour does not reveal your true desires, some authentic self hidden under the noise. It is quieter than that and more useful. It reveals what you have in fact been optimizing, which is not the same as what you most deeply want, and the difference between those two is exactly the thing worth finding. What you want is contested even to you. What you have been optimizing is a matter of record.

Which is why the method does not run through introspection. Asked to look inward and report your real motives, you will produce something, and it will be contaminated by all the same wishful pressures as the stated list, because it is the stated list wearing a more sincere expression. Your access to your own intentions is real but noisy, biased in your own favour in ways you cannot feel from the inside. So the method does not ask you to look inward. It asks you to look at the ledger, which does not care how you feel about it.

There is one advantage you hold here that no outside observer does, and it needs stating exactly. Earlier this book described a wall: you cannot infer what someone wants from their behaviour alone, because the same behaviour fits many different wants paired with many different mistakes, and nothing in the data tells you which. You do not escape that wall when you turn it on yourself. But you are standing on the lower side of it. You can set the record of what you did next to your memory of why, of what you were avoiding, of what the moment felt like, and while that memory is partial and self-flattering, it is more than zero. The gap between the clean record and the flattering memory is not noise. It is the most informative thing you have.

User's avatar

Continue reading this post for free, courtesy of Hugo.

Or purchase a paid subscription.
© 2026 Robonaissance · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture