Why did GPT-6 Astra score 62.7% and 99.9% on the same benchmark?

The two GPT-6 Astra numbers come from different harnesses. The standard harness is provider-neutral, so the model has to decide what to keep in visible notes between turns. The Provider Adapter harness lets Astra preserve opaque reasoning state across requests and compact its context, and that is the configuration where the score reaches 99.9%.

How much did the ARC-AGI-3 evaluation cost to run?

ARC Prize reports $26,098 for the 62.7% standard-harness run and about $19,000 for the 99.9% Provider Adapter run on the ARC-AGI-3 Semi-Private set. Human participants worked out to roughly $12.78 per attempted game, based on $115 per 90-minute session plus $5 for each game a player completed.

Does 99.9% on ARC-AGI-3 mean GPT-6 Astra has reached AGI?

No. ARC Prize states directly that saturating the benchmark 'would not represent proof of achieving AGI'. The organisation notes that ARC-AGI-3 environments have deterministic, closed-ended mechanics, unlike open-ended real-world problems. The result measures agentic exploration and planning inside those environments, and ARC Prize presents it that way.

How was the ARC-AGI-3 human baseline measured?

ARC Prize recruited about 500 general-public participants, not chosen for puzzle expertise, and each attempted roughly 9 games per session. The baseline for a level is the median action count among the players who completed it, so the comparison measures how efficiently a level was solved rather than whether it was solved at all.

ARC Prize · 2026-09-03 · major

GPT-6 Astra on ARC-AGI-3 — 99.9% with a provider adapter harness

ARC Prize published its measured results for GPT-6 Astra on the ARC-AGI-3 Semi-Private set: 62.7% for $26,098 through the standard harness, and 99.9% for about $19,000 through a Provider Adapter harness.

ARC Prize chart of GPT-6 Astra's action efficiency against the human baseline on ARC-AGI-3

ARC Prize's own run puts GPT-6 Astra at 99.9% on ARC-AGI-3, but only when the harness can carry the model's hidden reasoning state.

Key specs

Provider adapter run cost	~$19,000
Standard harness run cost	$26,098

Quick facts

Published by	ARC Prize
Benchmark	ARC-AGI-3 Semi-Private set
Model tested	OpenAI GPT-6 Astra
Harnesses compared	Standard (provider-neutral) and Provider Adapter (uses provider-specific features)
Action efficiency	Fewer actions than the human baseline on 96.0% of levels, 51.7% fewer per level on average
Human baseline	About 500 general-public participants, roughly 9 games per session
ARC Prize's caveat	Saturating the benchmark would not be proof of AGI

Benchmarks

ARC-AGI-3 (Semi-Private)

GPT-6 Astra — Provider Adapter harness		99.9%
GPT-6 Astra — standard harness		62.7%

source ↗

What is it?

ARC Prize has published its measured numbers for OpenAI's GPT-6 Astra on ARC-AGI-3, the benchmark that drops an agent into unfamiliar turn-based game environments with no instructions. On the Semi-Private set Astra reached 62.7% through the standard harness and 99.9% through a Provider Adapter harness. The distance between those two figures is the finding.

How does it work?

ARC-AGI-3 scores four things at once: exploration, modeling the environment, setting a goal, and planning the actions to reach it. The standard harness is deliberately provider-neutral — a model decides what to write into visible notes, which is what makes results comparable across labs. The Provider Adapter harness instead lets a model use vendor-specific features; for GPT-6 Astra that means preserving opaque reasoning state between requests and compacting long conversations.

Why does it matter?

Astra did not only finish the levels, it finished them cheaply in moves: fewer actions than the human baseline on 96.0% of levels, and 51.7% fewer actions per level on average. ARC Prize is blunt about the limit, writing that saturating the benchmark 'would not represent proof of achieving AGI' because these environments have deterministic, closed-ended mechanics. Money is the other catch — the two runs cost roughly $19,000 and $26,098, against about $12.78 per attempted game for a human player.

Who is it for?

people tracking agentic benchmarks

Frequently asked questions

Why did GPT-6 Astra score 62.7% and 99.9% on the same benchmark?: The two GPT-6 Astra numbers come from different harnesses. The standard harness is provider-neutral, so the model has to decide what to keep in visible notes between turns. The Provider Adapter harness lets Astra preserve opaque reasoning state across requests and compact its context, and that is the configuration where the score reaches 99.9%.
How much did the ARC-AGI-3 evaluation cost to run?: ARC Prize reports $26,098 for the 62.7% standard-harness run and about $19,000 for the 99.9% Provider Adapter run on the ARC-AGI-3 Semi-Private set. Human participants worked out to roughly $12.78 per attempted game, based on $115 per 90-minute session plus $5 for each game a player completed.
Does 99.9% on ARC-AGI-3 mean GPT-6 Astra has reached AGI?: No. ARC Prize states directly that saturating the benchmark 'would not represent proof of achieving AGI'. The organisation notes that ARC-AGI-3 environments have deterministic, closed-ended mechanics, unlike open-ended real-world problems. The result measures agentic exploration and planning inside those environments, and ARC Prize presents it that way.
How was the ARC-AGI-3 human baseline measured?: ARC Prize recruited about 500 general-public participants, not chosen for puzzle expertise, and each attempted roughly 9 games per session. The baseline for a level is the median action count among the players who completed it, so the comparison measures how efficiently a level was solved rather than whether it was solved at all.