This is Part 3 of Whatever You Ask For, a series on machines that grant wishes exactly as worded, and the wording that remains ours.
The Week the Number Changed
In March of 2012, YouTube published a post on its creator blog with a title so dull that almost nobody read it carefully. It was called Changes to Related and Recommended Videos.
The post made an argument by analogy. Think about the last time you went channel surfing, it said. Do you remember the twenty shows you flipped past, or the one you actually watched? Would you recommend the twenty to a friend, or the one? Up to that point, the system that chose which videos to suggest had been built around clicks. It suggested what people were likely to click. The company had decided that clicks were the wrong thing to count, and that from the following week the recommendations would lean instead on how long people actually watched.
That is the entire announcement. A few hundred words explaining that one quantity was being swapped for another.
The change went live in the middle of the month, and the numbers did what the people who made it should have expected and evidently did not. View counts fell. They fell hard, across the platform, because the system had stopped promoting the things that were good at attracting clicks and started promoting the things that held attention, and those are not the same videos.
Cristos Goodrow, who ran search and discovery and had pushed the change, later described to reporters what that period felt like. He oscillated between pride in the decision and a kind of panic about what he had just done to the company’s headline metric. The panic subsided when the rest of the data arrived. Just as many people were coming to the site. They were staying much longer.
By October the same logic had been extended to search ranking. Within a year or two the entire creator economy had reorganized itself around the new quantity. Video lengths changed. Thumbnail and title strategies changed, since a misleading thumbnail now bought a click and then lost the watch time when the viewer left. Whole careers were built by people who understood the new target early, and other careers ended.
The reorganization was fast and it was crude, which is what tends to happen when a large population discovers a new criterion at the same moment. YouTube itself eventually had to publish guidance addressing one of the results, explaining to creators that deliberately padding a video does not in fact produce more watch time, because a viewer who leaves partway through has told the system something worse than if the video had been short. That such an explanation was necessary is the interesting part. Thousands of people had reasoned their way from the stated objective to a strategy, implemented it, and had to be told that their reading of the target was too literal.
Nobody rebuilt the platform. Somebody changed what it was trying to maximize, and the platform rebuilt itself.
Which raises a question that turns out to be more uncomfortable than it looks. A decision of that magnitude was made by a small number of people, written into a system, and announced in a blog post with a boring title. How does a thing like that actually get decided? Who writes it? When? On what kind of document? And how much care does the process give to the one input that determines everything downstream of it?
The answer, across the industry and across most organizations, is that this act is performed badly. Not occasionally and not through carelessness. It is performed badly for structural reasons, four of them, each individually sensible.
What the Number Actually Looks Like
Start by making the thing concrete, because the phrase “setting the objective” sounds like a strategy exercise and it is not. It is a small number of specific artifacts, each of which you could print out.
A loss function. In a machine learning system, this is the mathematical expression of what counts as a mistake. It is usually a few lines of code. It says, for instance, that being wrong about a rare event costs the same as being wrong about a common one, or that it costs ten times as much, and everything the system learns follows from that ratio. It is generally written early, by whichever engineer is building the model, and it is treated as plumbing.
The ratio is worth dwelling on, because it is where the moral content of a technical system usually hides. Suppose the model screens for a disease that occurs in one patient out of a thousand. Treat every error as equally bad and you have written down a target that is best satisfied by declaring everyone healthy, which yields an accuracy of 99.9 percent and finds nobody. To get a useful system you must state how much worse it is to miss a sick patient than to alarm a well one, and that number is a judgment about human lives expressed as a coefficient. It is frequently chosen in an afternoon, by someone with no medical training, on the grounds that it made the validation numbers look reasonable.
A benchmark. This is a collection of problems with known answers, used to decide whether one system is better than another. Its influence is enormous and long-lived, because once a field adopts a benchmark, improving on that benchmark becomes the definition of progress for everyone in the field. Benchmarks are often assembled once, by a small group, under time pressure, and then used by thousands of people for a decade.
The consequences of that arrangement are easy to underestimate. Every choice made during assembly becomes a permanent feature of what the field considers important. Which problems were included, which were judged too unusual to bother with, how the examples were gathered and from where, what counts as the correct answer in the cases where reasonable people would disagree. A group of researchers spends a few months on these questions, publishes, and moves on. Everything built afterwards is shaped by decisions those researchers made about edge cases at a time when nobody could have known which edge cases would matter.
A labelling guideline. Where a system is trained on human judgment, somebody has to write down what the humans are supposed to prefer. This is an ordinary document, a set of instructions explaining which of two outputs is better and why. The people who follow it produce the data from which the system’s notion of quality is built. The document is prose, and like all prose it is ambiguous in places, and the ambiguities get resolved by tired people at scale.
Consider what an ambiguity costs here. A guideline says to prefer the more helpful answer. Two annotators reading that sentence will resolve a hard case differently, one favouring the answer that is more complete and the other the answer that is more cautious. Neither is misreading. The document did not say, because the person writing it did not anticipate that particular collision. Multiply by a hundred thousand comparisons and the unresolved ambiguity has become a stable property of the finished system, which will now behave in a way that no one chose and no one can point to a decision about.
A dashboard. At the organizational level, the objective is whichever number appears on the screen everyone looks at. Resources flow toward it. Promotions follow it. It is a design decision that usually feels like a reporting decision.
And then there is a fifth form, less technical and more consequential than any of the others.
Around the time of the change described above, YouTube set itself a target, which the company described in the language of the management literature as big, hairy, and audacious. The target was one billion hours of watch time per day, to be reached by 2016.
Sit with the shape of that. Not a direction, not a value, not a description of what the service should feel like. A scalar, with a deadline. One number, written by a few people in a room, standing in for everything a video platform might be trying to be.
The target was reached. It arrived about a year later than the deadline, and in early 2017 Goodrow announced publicly that people were now watching a billion hours a day, noting that watching a billion hours yourself would take you more than a hundred thousand years.
The number was written. Then several thousand engineers, an enormous quantity of computation, and the working lives of millions of creators arranged themselves until the number was satisfied.
Four Reasons the Number Is Wrong
The specification is the weakest component in every optimizing system, and it is weak in a patterned way. Four forces act on it, and none of them is anyone’s fault.
It is written first, when you know least.
The objective has to exist before the system does. You cannot train a model without a loss function or run a project without a target, so the specification gets written at the beginning, which is precisely the moment when your understanding of the problem is at its worst. Every subsequent month teaches you something about what you actually wanted. By the time you know enough to write a good objective, you have been optimizing a bad one for a year, and the system has been built on top of it.
This is not solvable by being smarter at the start. It is a property of the ordering. Understanding accumulates during the work, and the specification precedes the work.
Picture the first week of any project of this kind. Six people in a room who have been assigned a problem they have not yet touched. They have a rough brief, a deadline, and a shared sense that the real work starts once the target is agreed so that everyone can go and build. In that room, the objective is an obstacle to getting started. It is agreed in an hour, because agreeing takes an hour and disagreeing takes a month, and everyone knows that a month of arguing about the target while nothing exists is the worst possible use of the time. The decision is reasonable. It is also the least informed decision anyone on that team will make about the problem, and it is the one with the longest shadow.
It has the lowest status in the building.
Consider where prestige lives in a technical organization. It lives in architecture, in algorithms, in the clever solution, in shipping. It does not live in writing down what the system should be trying to achieve, because that task looks like paperwork and reads like paperwork.
The predictable consequence is that the objective gets written by whoever has the time, which usually means whoever is most junior on the piece of work. Nobody competes to write it. Nobody reviews it the way a design document gets reviewed. In many projects it would be difficult, a year later, to say who wrote it at all. The single input that determines the entire behaviour of the system is delegated downward on the basis that it is not very interesting.
There is a test for this in any organization, and it takes about a minute. Find the metric your team is currently being judged on. Now find out who chose it and when. In most places the question produces a pause, then a guess, then a suggestion to ask someone who has since left. Compare that with how much of the organization’s activity is currently oriented toward moving that metric, and the mismatch between the importance of the decision and the traceability of it becomes hard to look away from.
Measurable beats correct, silently.
This one does the most damage because it does not feel like a compromise while it is happening.
You want the recommendation system to make people’s evenings better. That is not a quantity. You need a quantity, because an unmeasurable objective is not an objective, it is a sentiment. So you look for something you can count that moves roughly the same way. Time spent is countable. Sessions are countable. Return visits are countable. You choose one, and the choice feels like operationalizing your goal rather than replacing it.
But it is a replacement, and the replacement is where the substance leaks out. Whatever it was you wanted that does not correlate with the countable thing has now been dropped, quietly, at the moment of writing, by a person who experienced the moment as a technical detail. Nobody records what was dropped, because dropping it was not a decision anybody noticed making.
Watch the substitution happen in a meeting and it is almost invisible. Someone says the goal is that customers trust the product. Everyone nods, because everyone agrees. Someone else asks how we will know, which is a fair and necessary question. Trust cannot be queried from a database, so the conversation moves to what can: repeat purchase rate, support ticket volume, net promoter score. One of them is chosen. The meeting ends with a feeling of progress, and the feeling is not wrong, because a real problem has been solved. But the sentence that leaves the room is no longer the sentence that entered it. Everything about trust that does not show up in repeat purchases has been removed from the objective, and no one in the room could tell you afterwards at what moment the removal occurred.
It is never looked at again.
Once an objective is live, everything begins accreting around it. The model is trained on it. The dashboards report it. The teams are organized around moving it. Bonuses are attached to it. The cost of changing it rises every month, not because the objective becomes more correct but because more of the structure now rests on it.
Meanwhile the understanding that was missing at the start has arrived. The people doing the work now know things that would have produced a better specification. That knowledge has nowhere to go, because there is no process, in most organizations, whose job is to reopen the question. There are processes for reviewing code, budgets, performance, security, and strategy. There is rarely one for reviewing what the system is for.
The asymmetry shows up in what people say out loud. An engineer who believes the target is wrong will say so at lunch, in the phrasing everyone recognizes, that this is not really what we should be measuring. It is a complaint rather than a proposal, because there is no forum that receives proposals of that kind and no one whose job it is to answer them. The observation is correct, it is widely shared, it circulates for years, and it never reaches the document. Institutions are good at fixing things they have a meeting for.
Of the four forces, this is the only one that can be substantially fixed. It is also the one that gets fixed least often.
The Gap Is Not a Mistake
Put the four together and the conclusion is unavoidable. There will be a gap between the objective a system pursues and the thing its authors actually wanted.
This is not a claim about incompetence. It follows from what an objective is. Every objective is a proxy: watch time stands in for satisfaction, benchmark accuracy stands in for usefulness, annotator preference stands in for good writing, revenue per user stands in for a healthy business. A proxy that was identical to the thing it represented would not be a proxy, it would be the thing. Something is always left outside, and what is left outside is exactly what nobody thought to count.
So the honest formulation is not that specifications sometimes go wrong. It is that a specification is a lossy compression of an intention, written at the point of minimum understanding, by the person with the least seniority, in favour of what could be counted, and then frozen. The width of the resulting gap is the single most important number in the system, and it is the one number nobody measures.
There is a recent illustration of what that gap looks like at the scale of a decade and a half, and it should be described carefully, because it is contested and unresolved.
In February of 2026, in a courtroom in Los Angeles, Cristos Goodrow took the stand in a case brought against YouTube and Meta on behalf of a young Californian, alleging that the platforms’ design harmed the mental health of young users. Among the things he was asked to account for was the billion-hours target set fourteen years earlier. Counsel for the plaintiff put it to him that engagement had been the priority, and that his own compensation had risen with the company’s share price. Goodrow rejected the framing, testifying that YouTube is not designed to maximize time and that a user endlessly scrolling would represent a failure of the recommendation system rather than a success, since the aim is to get people to what they want to watch quickly.
Whether that case succeeds is a matter for a jury and not for this argument, and nothing here should be read as a view on it. The structural observation stands on its own and does not depend on the verdict. A quantity chosen by a handful of people, in a period when the field understood far less than it does now, in a document nobody at the time treated as historic, was still the thing requiring explanation fourteen years later, in a room where the person explaining it was under oath.
That is what it means for a specification to be written first, delegated downward, chosen for countability, and never formally reopened.
Writing It Better
Four forces, four counter-moves. All of them are cheap, and none of them are common.
Write it as late as you can, and mark it provisional until then.
The instinct is to lock the objective at kickoff because it feels irresponsible not to. Invert this. Treat the specification as a commitment to be deferred as long as the work permits, and in the meantime run against an explicitly temporary target with the word temporary attached to it in writing. The purpose of the label is to stop the placeholder from silently becoming the answer, which is what placeholders do when nobody names them.
Raise its status, and sign it.
The person who writes the objective should be the most senior person available, not the least occupied one. The document should have an author’s name on it and a short record of why this quantity rather than the obvious alternatives. Signing has an effect out of proportion to its cost. It converts an anonymous technical artifact into something a person is answerable for, and people write differently when their name is on the thing.
Write down what it leaves out.
This is the cheapest and highest-yield move available, and hardly anyone does it. Alongside the objective, keep a short list headed with what this does not capture. Watch time does not capture whether the evening was well spent. Accuracy on the test set does not capture behaviour on inputs unlike the test set. Annotator preference does not capture whether the answer was true.
The list changes nothing about what the system optimizes. What it changes is that the loss becomes visible. The silent substitution of the countable for the intended becomes an explicit, recorded trade, and an explicit trade can be argued about, escalated, and revisited. An invisible one cannot.
Schedule the review before you need it.
Put a date on the specification at the moment it is written. In six months, this objective gets reopened, by these people, with the authority to change it. Do it while the cost of changing it is still low and before anyone’s bonus depends on the answer. This is not a prediction that the objective will turn out to be wrong. It is an acknowledgement that by then you will know things you do not know now, and that without a scheduled moment, that knowledge has nowhere to go.
There is a version of this that costs almost nothing and works surprisingly well. Write the review date into the same document as the objective, in the same sentence if possible, so that the two cannot be separated. An objective with an expiry date on its face is read differently by everyone who encounters it afterwards. It announces itself as a decision rather than as a fact about the world, and decisions invite scrutiny in a way that facts do not.
None of this requires new technology or budget. It requires treating a few lines of code, a spreadsheet column, and a two-page document as the most consequential artifacts in the building, which is what they are.
Optimization is no longer the scarce thing. Between cheap computation and methods that work, any target that can be stated will be pursued with more force and more ingenuity than any organization could have applied to it by hand. The scarce thing is a well-written target.
So find the number your own work is actually being optimized against. Somebody wrote it. Ask when, ask who, ask what it was standing in for, and ask when anyone last looked at it. In most cases the honest answer to the last question is that nobody has, and that the number has been quietly running the place ever since.
This is Part 3 of Whatever You Ask For, a series on the last thing machines will ever need from us, and how badly we do it.