Meta's AIRA₃ Beats 4,000 Human Teams to Win Kaggle Gold

Meta's FAIR team entered its next-generation autonomous research agent, AIRA₃, into a live public NVIDIA-run Kaggle competition and let it compete against human ML engineers. It won a gold medal, finishing 8th out of roughly 4,000 teams.

The task was concrete: fine-tune a 30B Nemotron model to reason better, with all competitors given the same information and scored on a hidden private test set. That external grading is what makes the result hard to dismiss. AIRA₃ was ranked by the same leaderboard as everyone else, not evaluated by Meta on a benchmark of Meta's choosing.

The lineup that won gold

AIRA₃ runs pairs of frontier LLMs inside coding harnesses. Meta disclosed exactly which pairings did what:

• Live gold medal (8th place): an ensemble of GPT 5.5 with OpenCode plus Claude 4.8 with ClaudeCode.
• Post-hoc gold: Muse Spark 1.2 with MuseCode.
• Post-hoc silver: Muse Spark 1.1 with OpenCode, and GLM 5.2 with OpenCode.

The post-hoc runs were graded on the same private test set, putting them on equal footing with the live entry rather than internal replays.

A shared forum instead of a central planner

AIRA₃ runs many long-horizon agents in isolated environments and lets them communicate through two shared substrates: a forum where they post hypotheses and findings, and a shared filesystem for solution artifacts. Search strategies emerge from that exchange. Each agent decides which of its peers' discoveries to build on, so exploration and exploitation are never scheduled by a top-down controller. Meta argues this is what lets the system convert additional compute into compounding knowledge rather than hitting the diminishing returns that plague long-horizon agent runs.

From benchmark to the wild

The predecessor, AIRA₂, was already the strongest open-source result on MLE-bench. According to that arXiv writeup, MLE-bench-30 spans tasks stratified into 5 low, 20 medium, and 5 high complexity, and AIRA₂ used asynchronous parallel execution plus a Hidden Consistent Evaluation protocol to prevent agents from gaming their own scores. All of that, though, was measured on a fixed academic benchmark.

AIRA₃ was validated against humans who had the same frontier tools available, on a task the system had never seen. That is much closer to what developers actually want to know: can an autonomous agent improve a real model as well as a strong human ML engineer working with identical resources?

Two other domains, same architecture

Meta tested generalization by swapping only the task specification and running the same agent design in two other settings:

• A 27% latency reduction on production GPU kernels on an internal benchmark.
• Gold-level performance on a separate Kaggle competition translating 4,000-year-old Akkadian clay tablets into English.

Kernel optimization and ancient-language translation share almost nothing with fine-tuning Nemotron. Reaching medal-level performance across all three by changing only the task prompt is a meaningful signal that AIRA₃ is not overfit to ML engineering specifically.

Caveats worth naming

Contamination is the first concern. Many top Kaggle solutions are public, and the underlying LLMs may have encountered similar problems during pretraining. The AIRA₂ authors acknowledged this directly: it remains difficult to determine how much improvement stems from genuine reasoning versus latent data retrieval. The Nemotron competition used a private test set, which reduces the risk for the headline result, but the general concern persists.

Cost is the second. AIRA₃ runs ensembles of frontier models with coding harnesses on top, and its predecessor already required 8 Nvidia H200 GPUs. This is not a lightweight autoML tool anyone will reach for to tune a small classifier.

Third, Meta has not released AIRA₃ itself. The earlier aira-dojo framework is open source, but the new generation exists so far as a blog post and thread rather than a repo or paper.

What the architecture actually implies

The winning configuration was an ensemble of off-the-shelf frontier models coordinated through a shared forum and filesystem, with no central planner, beating thousands of human teams using the same tools. That reframes what a research agent looks like in practice: less a monolithic model, more a small society of coding agents that compound each other's findings over time. If the pattern holds, the hard engineering problem for the next year is coordination substrates, not bigger single-agent scaffolds.