Whatever You Ask For

Machines now grant whatever we ask, better every year. Choosing what to ask was never their half of the problem. It is ours, and we are bad at it. Thirteen parts, one argument.

This is the overture to Whatever You Ask For, a series on machines that grant wishes exactly as worded, and the wording that remains ours.

---

The old wish stories all agree on one point, so completely that no telling ever bothers to state it. The genie can produce anything: palaces, armies, kingdoms. And the genie never picks the wish. All of the power sits on one side of the exchange. All of the choosing sits on the other. Every drop of drama in those stories lives in how the wish was chosen and how it was worded, because the granting was never in doubt.

We have now built the genie. The machines will pursue whatever objective is written into them, with enormous and still-growing capability, and with no view whatsoever on whether the objective was worth pursuing. The optimization half of every problem, finding the best route to a stated target, is being solved, faster and better than people can do it, and that is not a tragedy. It was never the hard half.

The hard half is deciding what the target should be. That job cannot be handed to the machine, not because the machines are not ready, but because of what the job is. And here is the uncomfortable part this series keeps returning to: we are bad at it. We write objectives in an afternoon and answer for them a decade later. We optimize numbers that quietly detached from the things they were supposed to measure. We do it to our companies, our schools, our children, and ourselves, and we mostly do not notice, because the choice arrives dressed as a fact.

Somebody decides what counts as success in every optimizing system, from a spreadsheet formula to a frontier model. This series gives that person a name, follows them through a century of machines, and ends with the question that cannot be delegated to anything: what should we ask for?

The Movements

Thirteen parts, in four movements. Each part stands alone. All of them live at the stage.

I. How Machines Learned to Optimize

Part 1 · Tell the Machine What, Not How: A cat in a puzzle box, a century of machines, and the moment we stopped writing instructions and started writing objectives.

Part 2 · Anything You Can Score, a Machine Can Win: What fell to the optimizers, and the quiet condition attached to every victory: someone had to make it scoreable first.

Part 3 · Somebody Wrote the Machine’s Goal in an Afternoon: Every objective has an author. One number, written quickly in 2012, was still being answered for under oath fourteen years later.

II. Why Objectives Go Wrong

Part 4 · The Machine Did Exactly What You Said: A program that pauses Tetris forever. A boat that circles instead of racing. What optimizers do with the letter of a goal.

Part 5 · Your Organization Was Reward Hacking Before the Machines Arrived: Goodhart’s law in institutions: the bank that optimized accounts opened, and the frictions that were removed one at a time.

Part 6 · Machines Can Learn What We Do, Not What We Want: Four ways to tell a machine what we want, and the residue every one of them leaves behind.

Part 7 · The One Step No Machine Can Take, and We Are Bad at It: The judgment that cannot be delegated, and the evidence that we exercise it inconsistently: two experts, one scan, different answers.

III. Where That Leaves Us

Part 8 · The Genie Never Picks the Wish: Why no amount of intelligence supplies a goal. The philosophical spine of the series, with the strongest objection given its full strength.

Part 9 · Your Worth Was Never Your Cleverness: Told through the people whose job title was once computer: what happens to worth anchored to a capability, and where the sturdier anchor was all along.

Part 10 · We Are the Species That Sets the Goals: A nerve that takes a five-metre detour, a process with no author, and the odd middle position we have always occupied.

IV. The Disciplines

Part 11 · You Have Never Read Your Actual Objective: The audit: what your hours, attention and money say you are optimizing, next to what you say you are.

Part 12 · We Are Training Children to Be the Machines: School as an optimizer factory: a decade of drilling on the one axis machines already own, and the ability no exam can test.

Part 13 · The Only Question Left Is What You Ask: No metric survives being made a target. What remains is a posture, four of them in fact, and one question this series leaves unanswered on purpose.

Five Doors

The parts were written to stand alone, so you do not need to begin at Part 1, and you do not need to read in order.

If you build or train systems, start with Part 4, then Part 6: what optimizers do to specifications, and why every method of writing down what we want leaves something out.

If you run a team or set targets for other people, start with Part 3 and Part 5, and read Part 13 before your next planning cycle.

If the capability jumps of the past few years have left you quietly wondering what remains yours, Part 9 was written for that feeling, and Part 8 is the argument underneath it.

If you are raising children inside an exam system, Part 12 is the one that will follow you around.

And if you only have one afternoon, read Part 11, which asks for nothing except the afternoon, and hands you the audit this whole series builds toward.

However you enter, the parts will keep pointing at one another, because they are all one argument: the machines will do whatever you ask of them, and the wording of the ask was the only part that stayed yours.

---

Whatever You Ask For: the last thing machines will ever need from us, and how badly we do it.