Somebody Wrote the Machine’s Goal in an Afternoon
A blog post nobody noticed. A number written for 2016. A courtroom in 2026. Every optimizing system has one human input, and it fails by design.
From Whatever You Ask For, a series on the one job machines cannot do for us.
The Week the Number Changed
In March of 2012, YouTube published a post on its creator blog with a title so dull that almost nobody read it carefully. It was called Changes to Related and Recommended Videos.
The post made an argument by analogy. Think about the last time you went channel surfing, it said. Do you remember the twenty shows you flipped past, or the one you actually watched? Would you recommend the twenty to a friend, or the one? Up to that point, the system that chose which videos to suggest had been built around clicks. It suggested what people were likely to click. The company had decided that clicks were the wrong thing to count, and that from the following week the recommendations would lean instead on how long people actually watched.
That is the entire announcement. A few hundred words explaining that one quantity was being swapped for another.
The change went live in the middle of the month, and the numbers did what the people who made it should have expected and evidently did not. View counts fell. They fell hard, across the platform, because the system had stopped promoting the things that were good at attracting clicks and started promoting the things that held attention, and those are not the same videos.
Cristos Goodrow, who ran search and discovery and had pushed the change, later described to reporters what that period felt like. He oscillated between pride in the decision and a kind of panic about what he had just done to the company’s headline metric. The panic subsided when the rest of the data arrived. Just as many people were coming to the site. They were staying much longer.
By October the same logic had been extended to search ranking. Within a year or two the entire creator economy had reorganized itself around the new quantity. Video lengths changed. Thumbnail and title strategies changed, since a misleading thumbnail now bought a click and then lost the watch time when the viewer left. Whole careers were built by people who understood the new target early, and other careers ended.
The reorganization was fast and it was crude, which is what tends to happen when a large population discovers a new criterion at the same moment. YouTube itself eventually had to publish guidance addressing one of the results, explaining to creators that deliberately padding a video does not in fact produce more watch time, because a viewer who leaves partway through has told the system something worse than if the video had been short. That such an explanation was necessary is the interesting part. Thousands of people had reasoned their way from the stated objective to a strategy, implemented it, and had to be told that their reading of the target was too literal.
Nobody rebuilt the platform. Somebody changed what it was trying to maximize, and the platform rebuilt itself.
Which raises a question that turns out to be more uncomfortable than it looks. A decision of that magnitude was made by a small number of people, written into a system, and announced in a blog post with a boring title. How does a thing like that actually get decided? Who writes it? When? On what kind of document? And how much care does the process give to the one input that determines everything downstream of it?
The answer, across the industry and across most organizations, is that this act is performed badly. Not occasionally and not through carelessness. It is performed badly for structural reasons, four of them, each individually sensible.
