NVIDIA's AVO Hits 100% on ARC-AGI-3 Where the Bare Model Scores 30%

NVIDIA just posted a result that reframes how much of an AI agent's performance comes from the language model versus the scaffolding around it. Their research system, called Agentic Variation Operators (AVO), cleared every level of the ARC-AGI-3 public set, a benchmark where the same underlying model scores about 30% on its own.

AVO completed the full 25-environment public set with a 100.00 RHAE score, solving all 183 levels in 6,624 environment actions. That's on a benchmark where Claude Opus 5 (High) is the highest-performing model on ARC-AGI-3, scoring 30.2% in a bare model evaluation. The jump comes from the harness, not a new model.

Why ARC-AGI-3 is a brutal test

ARC-AGI-3 is not a static puzzle set. It drops a model into unfamiliar, game-like environments with no instructions. No rules, no tutorial, no labeled goal. The system has to figure out what it's controlling, what the objective is, and how to get there, purely through interaction.

Scoring uses Relative Human Action Efficiency (RHAE), a metric that combines task completion with per-level action efficiency relative to first-time human baselines. Performance is aggregated across levels and environments. Solving a level is not enough; the agent has to solve it without burning wasted actions, so a good score requires both discovery and efficiency.

For context on how hard this is, when the benchmark launched, humans cleared 100% of the environments, and the best AI model at the time managed 0.37%. Even Opus 5's 30% was described by ARC Prize as a genuine reasoning leap, not benchmark hacking.

What AVO actually is

AVO is a general-purpose coding agent that treats long-running tasks as an outer loop rather than a single prompt. Like modern coding agents, AVO can inspect and edit code, run commands, consult documentation, and validate its work through execution. The novelty is what happens between steps.

The architecture wraps a frontier LLM in a four-step loop with two extra components:

• Inspect, plan, implement, evaluate as an iterative cycle over each candidate solution
• Persistent memory that carries forward prior implementations, evaluation results, compiler and profiler outputs, and accumulated reasoning, allowing the agent to resume from the current state rather than repeatedly reconstructing the search
• A supervisor that monitors the broader trajectory for stagnation or repeated unproductive cycles and can redirect the main agent toward alternative strategies when needed

The mental model is closer to an evolutionary search than a chat loop. Each iteration produces a candidate, execution feedback grades it, memory keeps the useful pieces, and the supervisor intervenes when the agent gets stuck exploring a dead end.

The GPU kernel result that came first

Before ARC-AGI-3, the team stress-tested AVO on something very concrete: writing faster attention kernels. In our attention-kernel study, AVO operated continuously for seven days, explored more than 500 optimization directions, and produced 40 committed kernel versions.

The output was not just an academic exercise. On NVIDIA DGX B200 systems, the resulting multihead attention kernels outperformed cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5% across the evaluated configurations. The agent subsequently adapted the evolved kernel to grouped-query attention in approximately 30 minutes of additional autonomous work. Beating hand-tuned FlashAttention-4 on Blackwell hardware is a nontrivial claim, and it happened without a human prescribing each step.

How the ARC-AGI-3 setup worked

Adapting AVO to ARC-AGI-3 mostly meant swapping the environment interface. The same agent, memory, and supervisor stayed in place. One choice stands out: the team went text-only.

In the AVO configuration, the LLM operated in a text-only modality: each observation was supplied as an exact 64 x 64 text grid, with no images or image tokens sent to the model. Consistent with the direct-interaction setup, the agent received the available actions without descriptions of the game's rules or goals and had to infer their effects through interaction.

That decision matters. VISTA's primary configuration uses a rendered 512 x 512 PNG, while also exploring textual-grid representations, so AVO's win is not attributable to a richer visual channel. It came from the agent loop itself.

How it stacks up against VISTA

The most useful comparison is with VISTA, another direct-interaction harness that also uses Claude Opus 5 as its backend. Both systems solve the same 183 levels, so the interesting question is efficiency.

AVO therefore used approximately 12% fewer actions in this cross-system comparison. This should not be interpreted as a controlled ablation: the two systems differ in agent backend, observation representation, memory, context management, and other implementation details. NVIDIA is careful about that caveat, but the direction is clear: better memory and supervision translate to fewer wasted moves.

The team also ran limited experiments swapping in GPT-5.6 Sol as the backend. In these limited experiments, Sol reached matched levels faster in wall-clock time in several cases, while Opus used fewer environment actions in matched-level comparisons. That suggests the harness is genuinely model-agnostic, with different frontier models offering different trade-offs.

The bigger claim, and what to be skeptical of

The paper's real argument is architectural. GPU kernels and ARC puzzles look nothing alike, but these results highlight that benchmark performance reflects the complete agent system, not only the underlying model. The same loop that iterates on CUDA code also iterates on hypotheses about invisible game rules.

A few caveats developers should keep in mind before extrapolating:

• The 100% is on the ARC-AGI-3 public set, not the semi-private or fully private competition sets. Those tend to be harder and more sensitive to overfitting.
• NVIDIA explicitly notes their run and the ARC Prize reference score used different reasoning settings and different agent setups, so the 30% versus 100% gap is not a clean isolation of AVO's contribution.
• The seven-day, 500-direction kernel run implies serious compute overhead. This is a long-horizon architecture, not a low-latency one.

Still, the pattern is what matters. If persistent memory plus a supervisor loop can move a Claude-tier model from mid-30s to a saturated benchmark score, the takeaway for anyone building agents is that harness design has not yet hit diminishing returns. The paper is available on arXiv for teams who want to look at the exact mechanisms behind the memory and supervisor components.