The harness is the product. GPT-6 Astra just proved it.

GPT-6 Astra launched on 3 September, and for about two hours the story was simple. New flagship model. 99.9% on ARC-AGI-3, the benchmark everyone agrees is the hardest one going. Greg Brockman called it the start of the AGI era. The number did the rounds and the takes wrote themselves.

Then ARC Prize published its own verification of the same model. Same benchmark, same week. Under their standard conditions, Astra scored 62.7%.

Both numbers are real. Nobody corrected anything, because nothing was wrong. The model didn’t change between the runs. The harness did.

We’ve been chewing on this all week. If you buy or govern AI systems, this is the most useful thing to happen to the benchmark circus in years, and almost nobody is framing it right. So let’s walk through it.

What ARC Prize actually ran

ARC Prize tests models two ways and reports both, which is the only reason this story exists.

The standard harness is their level playing field: a minimal, provider-neutral interface, so every model faces the identical test. Under it, Astra scored 62.7%. That run cost $26,098 at maximum reasoning effort.

The provider adapter is OpenAI’s own setup: the Responses API with the settings they chose, which keep the model’s reasoning state alive between steps and compact a long run instead of letting it overflow. Under that, 99.9%. $18,817, at high effort.

One wrinkle before you quote the 37-point gap at anyone. The effort settings differ between those headline runs. Matched at max effort, the adapter run hit 98.6% for $17,332. So the honest claim is this: same weights, same benchmark, same verifier, and 36 points of difference coming purely from the machinery wrapped around the model.

Notice something else. 62.7% isn’t a disappointing number. It’s the best standard-harness score anyone has ever recorded. Opus 5 sits at 30.2. GPT-5.6 Sol at 7.8. Astra is a strong model under both configurations, and that matters, because a weak model can’t be handed 36 points by configuration. The gap is only visible because the model is good.

What the adapter actually changes

So what is the machinery? Two things, and OpenAI says so itself, in a footnote to the result.

The first is persistence. In a long agent run the model reasons at every step: partial results, dead ends, working theories. The standard harness treats all of that as a transcript. When the context window fills, the earliest tokens get truncated and the run carries on without them. The Responses API harness keeps the reasoning state alive in a form the model can reuse, so step forty can actually build on what step twelve figured out.

The second is compaction. When context pressure builds, somebody has to decide what to keep, what to compress, and what to drop. That decision isn’t neutral. Good compaction preserves exactly the state the next step needs and throws away the rest. Do it well and the model behaves like it never forgot anything. Do it badly, or don’t do it at all, and the same model trips over work it did ten minutes ago.

Two settings. That’s where 36 points live.

And to be clear, this isn’t a scandal. ARC Prize built the standard harness on purpose, arguing that a genuine general system should solve hard tasks through a minimal interface. OpenAI disclosed which configuration produced its number. Everyone behaved, and the numbers still land in different postcodes. Good faith isn’t the issue here. Ownership is.

Why this should change how you buy

If your procurement treats the model as the product, you now have a structural problem. The number in the deck describes a system. Part of that number is the weights. Part of it is a configuration the vendor controls, operates, and can change without asking you.

So three questions, and most benchmark coverage never asks any of them.

Who owns the harness? The harness decides what the agent remembers, which tools it can reach, how it recovers from a failed step, and what a run costs. Whoever owns it makes those calls. If your contract is for model access, that isn’t you.

Where does the harness run? The best measured configuration of Astra only exists inside OpenAI’s boundary. A week after launch, OpenAI put the harness itself up for rent as the Agents API. The terms: US data residency, no zero data retention. For plenty of work that’s a fair trade. For a ministry, a bank, or a defence programme, it’s the exact exposure your procurement process exists to prevent. The 99.9% configuration is not available in your jurisdiction. The model is. The number isn’t.

What does the harness log? The reasoning state that produces the 98.6% is opaque by design. You can’t inspect it. Neither can an evaluator. That’s the opposite of an audit trail, and it’s the layer that would have to answer when a regulator asks why your system produced a particular answer at a particular moment.

None of this makes Astra a bad product. It makes the benchmark number a property of a system you wouldn’t be buying.

The bit the launch coverage mostly missed

Here’s the thing. Nothing in that adapter is magic, and nothing in it needs to happen in someone else’s cloud.

Persistence is a runtime decision. Compaction is a runtime decision. Tool scaffolding matched to the model is a runtime decision. All three can be made inside your boundary, on your hardware or your tenancy, under your jurisdiction. You give up the vendor’s tuning and gain the ability to tune for your own workloads, which the vendor was never going to do for you anyway.

This is the layer we build. A governed runtime where the harness is the thing you own. The state that persists between steps is inspectable, because it has to satisfy your auditors rather than a benchmark footnote. Every action is signed by a named identity and approved per policy. Spend is capped before it happens, not reported after. And the model slot is swappable, so the harness you tuned for Astra today runs K3 or a MIT-licensed DeepSeek Flash at a fraction of the price tomorrow, with your controls untouched.

The 36 points don’t have to come bundled with someone else’s cloud.

Four questions for the next deck

ARC Prize did everyone a favour by publishing both numbers instead of just the flattering one. The score belongs to the system, and the system is what you’d actually be running. So the next time a benchmark number lands in a deck, ask four questions before it gets anywhere near a business case.

Which harness produced this number: a provider-neutral setup, or the vendor’s own?

Who operates that harness in production, and where does it run?

What state does it keep between steps, and can my auditors see it?

What happens to the number, and to the cost, when the model underneath gets swapped?

A vendor who can answer all four clearly is selling you a system. One who can’t is selling you a screenshot of someone else’s.