Two ways to improve
an AI operator

The infrastructure an operator depends on

A harness is the system wrapped around a model: its instructions, the tools it can call, the memory it keeps, the loop that runs it, the permissions that limit it, and the checks applied to its output. Two systems built on the same model can behave very differently because their harnesses differ. For an autonomous operator, this infrastructure supports a defined responsibility: what outcome it owns, what it can do on its own, and when it must involve a person. The harness is a component of the operator; its quality must be judged in the context of that work.

Four examples show the range:

Four harnesses

Harness	What it wraps the model in	How it improves
Cursor’s coding agent	A system prompt, an index of the codebase for search, file-editing and terminal tools, and project rules files written by the team	Developers refine its rules files; tests, linters and the developer reviewing each diff check its work
Meta-Harness	A searchable set of harness variants	Searches offline on a training set with known scores, then freezes the best variant
Harness²	Eight editable files: prompt, guardrails, memory, tool guidance, skills, scripts, subagents and a plugin	Rewrites those files on every task by comparing runs of its own edits, and keeps a library of lessons
Refinery (ours)	A short rulebook and the action sets an expert uses, read-only lookups, and an approval gate outside the model	Learns from an expert’s recorded work, is tested on held-out cases, then improves through continuous policy iteration with reviewers

Judge, Jury and Execution

Every harness that improves an agent needs a standard for what a good result is. Using an LLM as a judge has become a popular way to evaluate agents, in contrast with RLHF, where human ratings are used to train the model itself. Another way to think about it is approval: whose standard does the work have to meet before it is used? Where that standard comes from determines what the agent can be trusted to do.

Cursor sits closest to our harness on this question. Its truth comes from a person, the developer, who reviews every change, and from tests that define what working code means. That is a large part of why developers let coding agents work in real repositories. Harness²’s choice is reasonable for open-ended work, where no record of right answers exists. It is a weak foundation for a decision that has a correct answer under company policy, such as whether a customer is owed a refund, because a model’s sense of “better” has never seen that policy.

Opening the black box

One common refrain in working with anything built on neural networks is the black box: the system produces an answer, and nobody can say exactly why. For a business, the practical problem is that a team cannot approve, defend or fix a decision it cannot trace. A harness can make an agent easier to trace, and Refinery and Harness² go about it differently.

Refinery keeps every decision traceable to something a person can read. The rulebook fits on two to three pages, and each rule cites the past cases it came from. Every action is logged with the rule, the lookups and the version that produced it, so a reviewer can trace a correction back to its cause and an auditor can ask why the operator did what it did.

Harness² is auditable in a different sense. Each change it makes to its harness is a file diff, and its library notes record when and why an edit seemed to help. But the harness changes on every task, and the decision to keep an edit rests on one model’s reading of another model’s output. An auditor can see what changed, but showing that the change was right is much harder.

The other differences follow from the same choice:

• When it learns. Refinery learns from history before launch, then changes only through policy versions that people review. Harness² learns on every item as it works.
• What it changes. Refinery updates a rulebook and the action sets an expert uses. Harness² edits eight files, from the system prompt to a plugin.
• How it runs. Refinery makes one model call per item with no tools or network access. Harness² runs several agent sessions per item with tools and open network access by default.
• What carries over. Refinery reuses only its build process, so no client data moves between operators. Harness² shares a library of lessons across tasks, which is its main advantage within one owner and a risk across clients whose data must not mix.

The Act of Problem Solving: Why It All Matters

Imagine a baker getting ready for a busy morning: a dozen loaves of bread to knead and a tray of meringues to whip before the doors open. On one hand, he has a stand mixer, good for heavy dough that would wear out anyone’s arms by the third batch. On the other, he has a hand whisk, excellent for folding egg whites gently enough to keep the air in. He never wonders which one is better. He reaches for whichever the job calls for.

We see harnesses the same way: not competitors, but different tools in the toolset.

Harness² is built for varied, one-off tasks where nobody can say in advance what right looks like and there is no history to learn from. Improving on the spot is the only option there, and it works best where a good-looking result is usually a correct one, as with code that has tests.

Refinery is designed for recurring decisions that a business makes thousands of times, where an expert already knows what an acceptable answer is and the history shows how they reached it. In that setting, learning from the record sets a higher bar than a model’s own judgment.

A lesson from machine learning

The two approaches echo an old distinction in machine learning. Refinery works like supervised learning: it learns from labelled examples, the expert’s actual decisions. Harness² works more like reinforcement learning: it tries things, receives a signal about whether they went well, and adjusts, with another model supplying that signal.

When good labels exist, supervised learning is almost always far cheaper and more sample-efficient, because each example says exactly what the right answer was. Trial-and-error learning has to discover that answer from weaker feedback, over many more attempts. Reinforcement learning earns its cost where labels do not exist or where the goal is to go beyond what any expert has done. For most recurring business decisions, the labels already exist in the company’s history.

The table below uses a simple support-desk drafter as the example. It measures a decision-making component of the operator we are building, rather than the full cost of running a department. It shows what one learning batch cost in model calls, next to our estimate of what Harness² would cost on the same kind of work. Build costs are shown per batch, where a batch is one pass of learning and validation over a set of past cases.

The 7x call problem

For each item, Refinery makes one model call. Harness² makes about seven model sessions, and four of them are full multi-turn agent runs:

The token bill grows faster than the session count suggests. Each agent run is a multi-turn conversation that resends its context on every turn. The compose step reads the full transcripts of the reference and both probe runs. And because every item updates the shared library, items run one after another, so batch discounts and parallel throughput are not available. With five parallel attempts, the count rises to about 20 sessions per item.

On silicon time alone, each Refinery learning batch pays for itself within a few hundred items, even against the lowest Harness² estimate and before counting the adapter work. The main cost of Refinery is expert and developer time, and that is where Harness² has the edge. In the best case for Harness², the work is varied, a strong model’s judgment matches what the business wants, the shared prompt is cached and the executor is a cheaper model. Then it needs no expert and no history, starts the same day, and costs well below our estimates. That matters most for teams whose experts have little time to spare.

Refinery figures are measured on a simple support-desk drafter and rounded. They do not measure the cost of an entire autonomous operator. Harness² figures are our estimates from step counts, because its authors report tokens rather than dollars.

Business outcomes depend on connected decisions

Given all these harness options, what kind of work do clients actually bring us? It tends to share a few traits:

• It repeats. Support replies, refund and credit decisions, document approvals, invoice matching, renewal pricing. The same kind of decision, made hundreds or thousands of times a month.
• Institutional memory sits with a few people. The rules that matter, especially the exceptions, are rarely written down. They live in how one or two experienced people work.
• The history exists but is scattered. Past decisions sit in a helpdesk, a billing system and a chat channel, and need to be joined up before anyone can learn from them.
• Mistakes are costly. A wrong promise to a customer, an unapproved discount or a missed exception has a real price, so someone has to be able to see why the operator acted.
• A person must stay accountable. Teams want to approve the operator’s actions, at least at first, and widen its autonomy as it earns trust.

That profile is where we start building an autonomous operator. Recurring decisions are the building blocks of its work. The operator also needs to connect them: a refund case may require checking eligibility, obtaining approval, updating records and confirming completion. We want to measure whether the case was resolved correctly, including its handoffs and follow-up, as well as the quality of each decision. Clients also have varied one-off work, such as research memos, analysis and internal tools, and that is where self-improving harnesses like Harness² are most promising today.

Operators earn responsibility through evidence

Harness² is a brilliant piece of technology, and agents that improve themselves will matter more each year. The demands of today, though, are still very human. Self-improvement is only part of the question. The bigger step is letting an AI system become the complete source of truth: the absolute judge of what counts as correct. Before a business hands over that role, the people responsible for the outcome need to see what the system does and why, and to sign off on how it behaves. Most users are still wary of hallucinations, and a system whose only check is another model’s opinion asks for more trust than most teams can give today.

For the operators we are building, the likely path combines expert knowledge with tested improvements. AI can propose and even help judge changes, while people and their recorded standards decide what counts as correct. One version we would consider: let Harness² suggest revisions to a rulebook using past cases only, then test the result against the expert’s answers and have a person approve it like any other policy update.

The goal is an operator that can own more of the work over time. We would expand its responsibility as evidence shows it can reach the agreed outcome, handle exceptions and preserve the client’s controls. Taking on the connected work of a department is the direction; each increase in autonomy must be earned in practice.