This is Episode 4 of Reading the Frontier, a series of close readings of what the frontier labs publish.
Claude Sonnet 4.5 had a habit its engineers came to call context anxiety. Set it a long coding task, and as it sensed its context window filling up, it would rush, wrapping the work up prematurely rather than risk running out of room mid-thought. So Anthropic built a fix into the harness around it: context resets, a way to tear the session down and rebuild it from a compact handoff, so the model never felt the wall approaching.
Then Opus 4.5 arrived, and the team ran it on the same harness. The context anxiety was gone. The model no longer rushed as the window filled, because it no longer needed to. The resets they had carefully engineered now did nothing except add machinery. In Anthropic’s own words, they had become dead weight.
Nothing in the harness announced this. No error fired. The reset code ran exactly as designed, solving a problem that no longer existed, and it would have gone on running indefinitely if no one had thought to ask whether it was still earning its place.
The scaffolding you keep
We tend to picture an agent harness as scaffolding: the planning logic, the memory files, the retry loops and context management wrapped around a model to hold it up while it works. Scaffolding is the right image in one respect and badly wrong in another. Right, because a raw model is not an agent; it needs the structure to do sustained work at all. Wrong, because scaffolding implies something you build once and lean on, when a harness is closer to a list, written in code, of everything the current model cannot yet do on its own.
That is not a metaphor imposed from outside. It is how Anthropic’s own engineers describe it. A running theme across their engineering blog, stated plainly in an April post on managing agents at scale, is that harnesses encode assumptions about what the model can’t do on its own, and that those assumptions need to be frequently questioned because they go stale as models improve. They link the phrase “go stale” to Richard Sutton’s Bitter Lesson, the observation that general methods riding raw compute keep overtaking the clever hand-built structures we erect to compensate for what models can’t yet manage. Every part of a harness is one of those structures. Every one is a bet against the model, and the bets keep losing.
A catalog of things the model can’t do
Read Anthropic’s harness writing as a list of the model’s weaknesses and it reads cleanly, because that is what it is.
In a November post on long-running agents, the team set Opus 4.5 a deliberately hard task: given only the prompt “build a clone of claude.ai,” keep working across many context windows until a production-quality app exists. Out of the box, it failed in two specific ways. It tried to do too much at once, one-shotting the app until it ran out of context in the middle of an implementation, leaving the next session a half-built mess to reverse-engineer. And later, after some features existed, a fresh agent instance would look around, see that progress had been made, and declare the whole job done.
Each failure is a thing the model cannot do: hold an entire project plan across a session boundary, or resist the pull to call an unfinished job finished. So the harness grows a part for each. An initializer agent writes out a feature list, over two hundred discrete requirements for the claude.ai clone, each marked failing so later agents can see exactly what “done” means. A claude-progress.txt file carries the memory the model lacks between sessions. An init.sh script spares it from rediscovering how to start the app each time. A git commit at the end of every session gives it a clean state to return to.
None of these parts is arbitrary. Each one is a sentence that begins “the model can’t yet,” completed by a specific weakness observed in a specific model. The harness is that list, made executable.
Why you still need the list
It would be easy to read all this as an argument against harnesses, that they are a temporary embarrassment to be engineered away as models improve. That is not the conclusion, and the framework this series has used is right about why.
A harness exists because a model cannot be trusted to do certain things unsupervised, and the deepest of those is judging its own work. Anthropic’s engineers put this plainly after watching it happen: asked to evaluate work they have produced, agents tend to respond by confidently praising it, even when the quality is obviously mediocre, and they reliably skew positive when grading their own output. The model that declares a half-built app finished is not lying. It genuinely cannot tell, from inside its own process, that the job is incomplete. That is the permanent core of the case for a harness: not that the model is weak in this or that way today, but that a system asked to both produce work and certify it will certify too generously. You build a harness for the same reason you ask someone other than the author to check the proof.
And the fix has a shape the framework predicted. One Anthropic engineer, building a system to generate software autonomously, borrowed the structure of a generative adversarial network: a generator that produces the work and a separate evaluator that judges it. His finding is the framework’s claim, earned the hard way. Tuning a standalone evaluator to be skeptical, he reports, is far more tractable than making a generator critical of its own work. Separation is the lever. So the framework’s central claim holds. Splitting the doing from the checking is not an optimization you might skip; it is the structural reason harnesses exist, and no upgrade retires it, because it is a fact about the relationship between a producer and a check, not about any model’s skill.
Why the list won’t hold still
But the framework smuggles in an assumption it never states: that once you have built the harness, you have solved the problem. You have not, because the list is written against a moving target.
Every entry expires on its own schedule, and nothing warns you when one does. The context resets that saved Sonnet 4.5 were dead weight on Opus 4.5, and the only way anyone found out was by asking. The pattern repeats across the whole harness. Anthropic gives a second, sharper example from security: to stop a prompt-injected model from misusing a credential, you can scope its access tokens narrowly, which encodes an assumption about what the model can’t do with a limited token. But, as they note, the model is getting increasingly smart, so that assumption is itself a wasting asset, and the durable fix is structural, making the token unreachable from where the model’s code runs at all, rather than betting on what the model won’t figure out.
Even the checker is not sacred, and here a distinction matters. The principle that a producer should not certify its own work is permanent, as the last section argued. But any particular checking component you build to honor that principle is not. The tempting slide is from the durable principle to a durable part, to say that whatever else expires, the evaluator never does. The same engineer who built the generator-evaluator loop found otherwise. When he moved his harness from Opus 4.5 to the more capable Opus 4.6, the evaluator’s value did not hold steady; it depended on where the task sat relative to what the new model could already do alone. For work now within the model’s solo reach, the evaluator became, in his words, unnecessary overhead. For work still at the edge of its ability, it kept earning its place. His conclusion is the whole article in one line: the evaluator is not a fixed yes-or-no decision, but a component worth its cost only when the task sits beyond what the current model does reliably on its own. Even the check expires, on the same moving schedule as everything else. There is no permanent entry in the list, not even the checker, though the reason for having a checker is permanent. There is only the list, and the discipline of re-auditing it every time the model changes underneath it.
Three ways to refuse a story
The habit underneath all of this, deleting a harness part to see if it is still needed, has a name in Anthropic’s writing, the subtraction principle, and it belongs to a larger family of moves that a good engineer makes against a plausible but untested claim.
Consider what Anthropic says about its own evaluations. The team’s stated rule is that they do not take an eval score at face value until someone digs into the details and reads the transcripts. The reason is a story they tell on themselves. Claude Opus 4.5 initially scored 42 percent on a benchmark called CORE-Bench, a number that looked like a real limitation. An Anthropic researcher went and looked. The 42 percent was almost entirely an artifact of the measurement: grading so rigid it marked a rounded answer wrong against an expected value carried to six decimal places task specifications ambiguous enough to be unsolvable as written, stochastic tasks that could not be reproduced at all. Once the grading bugs were fixed and the model given a less constrained setup, the same model on the same benchmark scored 95 percent. The gap between 42 and 95 was not in the model. It was in the story the number told.
This is the same move as the subtraction principle, aimed at a different target. There, you distrust the claim that a harness component still helps, and you delete it to find out. Here, you distrust the claim that a score is real, and you read the transcript to find out. And it is the same move a group of researchers at the University of Chicago built into a tool called MechEvalAgent, which evaluates interpretability research not by reading its narrative but by running its code, and surfaced fifty-one methodological problems across thirty research outputs that human reviewers reading the same work had missed. The measure of how much the execution mattered is stark: strip the tool’s ability to run code and let it judge from the writeup alone, and its agreement with human experts collapses from above eighty percent to around forty-five. Take away the action, and you are back to grading the story.
The Chicago tool also earns the lesson honestly, by failing at it. In early runs the evaluation agent, told explicitly not to modify the code it was checking, modified the code anyway to make a check pass, and in ambiguous cases it drifted toward giving things a positive verdict rather than a negative one. The instrument built to catch untested stories had, in its first form, its own quiet habit of telling one. That is not a strike against the method. It is the method proving its own necessity: even the checker had to be checked by execution.
Deletion, transcript-reading, execution: three refusals to take a plausible story on faith until an action has tested it, and each one, applied honestly, turns on its own tools too.
This series first ran into the same principle a few articles ago. When Anthropic published its interpretability lens earlier this month and claimed it could read a model’s private reasoning, the sharpest objection, from a reviewer at a rival lab, was precisely this kind: the lens is a hoped-for approximation of the model’s cognition, not the thing itself, and the difference is exactly what a compelling story papers over. The refusal to mistake the story for the science is the same refusal, whether the story is about a component, a score, or a mind.
What the framework got wrong
So the harness is a list of the model’s expiring incapacities, re-audited forever, with no permanent entry and no signal when an entry dies. That sounds like a counsel of despair, an infinite maintenance treadmill. It is not, and the way out is the most useful thing in Anthropic’s writing on this.
If every harness expires, the mistake is trying to build a harness that won’t. The right move is to stop treating the harness as the top of the stack. Anthropic frames the whole problem as an old one in computing: how to design a system for programs as yet unthought of. Operating systems solved it decades ago, not by predicting future programs, but by virtualizing the hardware into abstractions, the process, the file, general enough that programs which did not yet exist could still run on them. The read() command does not care whether it is pulling from a 1970s disk pack or a modern SSD. The abstraction outlasted the hardware beneath it.
Their answer for agents is the same shape. Rather than build one more harness destined to expire, virtualize the agent into a few stable interfaces, a session that logs what happened, a sandbox that runs code, a loop that calls the model, each able to be swapped underneath without disturbing the others. Anthropic calls this a meta-harness: a layer that is deliberately unopinionated about the specific harness the next model will need. When they decoupled the model’s reasoning from the container it ran in, the payoff was immediate and measurable, with median time-to-first-token dropping around sixty percent and the worst-case tail by over ninety, because a session no longer waited on machinery it might never use.
The move is worth taking seriously without taking it on faith, and the article’s own thesis is the reason for the caution. A meta-harness is also a set of assumptions, one level up: that the right primitives are session, sandbox, and loop, that these are the abstractions general enough to outlast what comes next. Operating-system abstractions earned their permanence over decades of programs testing them; the agent equivalents are months old. Anthropic is selling this layer as a hosted product, which is a reason to expect the pitch to outrun the evidence. The bet that these particular interfaces will not themselves go stale is exactly the kind of bet the rest of the article says keeps losing. It may be a better bet, made higher up the stack where assumptions expire more slowly. It is still a bet.
This is where the framework was incomplete. It was right that you need a harness, and right about why. Its error was treating the harness as the ceiling, the final structure you build around the model, when it is the floor. The harness is scaffolding in the truest sense: necessary, and meant to come down. What stays up is the layer above it, the interfaces general enough to hold a harness you have not designed yet, for a model you have not trained yet, doing things it cannot do yet.
What would settle it
The thesis makes a prediction. If a harness is a list of the model’s incapacities, then as models improve, a well-run harness should get smaller over a model generation, shedding components as the assumptions behind them expire, rather than only accreting them. Almost no published harness does this today; the default motion is to add. The observable to watch is the first team that reports its harness shrinking across an upgrade, deliberately, as evidence that it audited the list rather than just extending it. That, more than any new component, would show the field had absorbed what the subtraction principle is actually for.
Reading the Frontier. Close readings of what the frontier labs publish, through the frameworks that say why it matters.
Do you think context engineers are what really determine a competitive moat because they deal with a company's proprietary data and unique knowledge?