Deep dive: Automated harness tuning: is the agent wrapper still a moat?

Why today

Earlier this month, researchers at Google Cloud AI Research and Northwestern University posted a paper called Harness squared. They released the code on GitHub under the Apache 2.0 licence. The idea is easy to state. Keep the agent's model frozen, and let a second, stronger model rewrite everything around it: the prompts, the tools, the memory, the guardrails. The headline result is a gain of up to 11.5 points on a workplace agent benchmark called WorkBuddy-Bench. On a legal agent benchmark from Harvey, the best gain is 11.8 points. Both numbers come from the authors. The score matters less than the licence. Many agent startups have built quietly on hand-tuning the wrapper around a model. If a free library can do a decent version of that work automatically, the question for builders and investors changes. What is left that a frontier model, editing your harness overnight, cannot reproduce?

Plainly

Start with the word harness. In the paper's words, it is "the prompts, memory, skills, tools, and orchestration through which the model performs tasks." The authors break it into eight editable parts. These include the system prompt, guardrails, memory, skills, tool guidance, scripts, subagents and a middleware plugin. Each part is a file, so another model can edit it. Picture a test kitchen. A line cook follows a recipe card. The cook is the executor model, and the card is the harness. In the paper, the cooks are GLM-5 and Gemini 3.5 Flash. The head chef is the editor, here Anthropic's Claude Opus 4.8. The method has three steps: propose, probe, compose. First, the chef watches the cook make the dish once with the current card. Then the chef writes two revised cards. One fixes general habits, like skipping verification steps. The other fixes things specific to this kind of dish. That is the propose step. Next, the cook makes the same dish twice more, once with each new card. That is the probe step. Now the chef compares three plates side by side. There is no customer rating and no answer key. The authors call this verifier-free. The chef judges from the differences in how each attempt went. Then the chef writes a final card. It might merge the two edits, pick one, or keep the original. That is compose. There is a second loop on top. After each task, the chef writes lessons into a library, a file called EXPERIENCE.md, so the next dish starts from a better card. The authors call this harnessing-level recursion: improving how future improvements get made. Two practical details matter. First, this happens while the task is being done, on the task itself. Second, the default setting takes one reference run, two candidate runs and a composition step. That is roughly four executions where a plain agent uses one, plus the frontier editor's tokens on top.

What the evidence shows

Here is what is established. The code is public and the method is described in enough detail to rerun. The benchmarks come from outside the team. Tencent's WorkBuddy team built WorkBuddy-Bench. Grading mixes hidden tests and file checks with model-based judges, and Gemini 3.5 Flash serves as the judge. The headline number needs context. The 11.5-point gain is the average for GLM-5, the weaker executor. With Gemini 3.5 Flash, the same benchmark improves by 3.1 points. On Harvey's legal benchmark, GLM-5 gains 11.8 points while Gemini gains 2.0. The pattern is consistent: big lifts for weaker models, small ones for stronger models. The authors also ran the strongest model as both editor and executor. In that setup, Claude Opus 5 gains 3.3 points on WorkBuddy-Bench and 3.0 on the legal benchmark. Against Meta-Harness, the strongest baseline and an offline search method, Harness squared wins on WorkBuddy-Bench in all four configurations. Meta-Harness leads on one legal configuration, 76.5 to 75.6. Now what is claimed but not shown. The text we read gives no per-task cost in tokens or dollars. One figure counts total tokens, but no dollar figure is attached. The paper does include a best-of-k baseline, which simply runs the base agent several times and picks the best result. We could not find a clean side-by-side against it at equal budgets. Our reading of the paper is that baselines are not compute-matched. That matters. Sayash Kapoor and colleagues at Princeton showed that models are stochastic, so extra calls alone raise accuracy. They also found that simple baselines beat complex agent designs on HumanEval at much lower cost. Then there is the editor confound. When Opus 4.8 rewrites a GLM-5 harness, some of the gain may just be a stronger model's judgement copied into text. DSPy and GEPA, the best-known prompt optimisers, are discussed but not run as direct competitors. GEPA's own paper reports beating reinforcement learning with up to 35x fewer rollouts. Whether any of this carries over to production is not shown.

How people read it

The same facts support different readings. The AGI-forward reading focuses on the second loop. A system that improves its tools without retraining, and also improves how it improves, is an early form of recursive self-improvement. Gains held across two model families and three benchmarks without weight updates. Add better editors and longer task streams, and the improvements could compound. The AI-realist reading sees mostly a transfer of skill. The gains shrink sharply as the executor gets stronger, to about 3 points for Opus 5 on WorkBuddy-Bench. Each task costs several runs plus frontier editing. And grading partly depends on a model judge. On this view, the method is a clever way to apply a big model's judgement to a small model's work at inference time, paid for every task. The market reading sits in between. The largest gains appear for cheaper open models like GLM-5. If that holds up on real workloads, it is a cost lever. Teams could run a cheap executor and buy frontier judgement only when it changes the result. That would be good for anyone shipping agents on open or mid-tier models, and bad for anyone selling tuning as a service. One more detail is worth noticing. A Google paper used an Anthropic model as its editor. Google appears to be giving the editing method away rather than tying it to its own models. Which reading wins depends on numbers nobody has published yet: dollars per task, at matched compute.

What it means for builders and investors

Our read, held provisionally, is that hand-tuned prompts and tool wiring are no longer a moat on their own. They were always fragile, and they now look automatable. What an automated editor cannot easily get is ground truth. Harness squared is verifier-free by design because it lacks an answer key. A company that has real outcome data has something the paper does not. That might be accepted contract redlines, resolved support tickets or passing production tests. The same goes for proprietary evaluation sets, deep integrations into customer systems and the trust that comes with owning a workflow end to end. Distribution and pricing count as much as engineering here. A team that owns the customer relationship can adopt this library tomorrow. A team whose edge was the tuning cannot easily get that edge back. One question worth exploring in diligence might be this. Suppose a competitor ran an open harness optimiser with a frontier editor on your tasks for a month. How much of your lead would survive, and what data would that optimiser never see? We would also want to see a company's own evaluation set and its cost per completed task. A founder who can show both probably understands where their edge really comes from.

What to watch

Three things would settle this. The first is an independent replication with fair baselines: best-of-k at the same number of executions, the same frontier editor across methods, and cost per task in dollars. If the gain survives that, the method is real. The second is results with strong executors on long, real workloads. If gains stay at a few points for frontier models, this is a tool for cheap models, not a general unlock. The third is adoption. Watch whether the major agent frameworks build harness search in as a default feature. Review of the paper and early community reproductions should give a first signal within a quarter or two. Production evidence will take longer.