Prime Intellect's Fable 5 Closes 82% of the Human AI Research Gap
Prime Intellect ran 153 autonomous research runs across 18 frontier models on the nanoGPT optimizer speedrun, with the best closing 82% of the gap to a record built by dozens of humans over months.
- Prime Intellect ran 153 autonomous research runs across 18 frontier models on the nanoGPT optimizer speedrun, the largest open experiment of its kind.
- Best runs closed 82% of the gap to a human record built by dozens of researchers over months; Fable 5 led at 81.7% gap closure (2,726 steps).
- Top models succeeded by selecting the right experiments, navigating benchmark noise, and revisiting old negatives as the recipe evolved.
- Key finding: frontier agents excel at optimizer search and hyperparameter sweeps but struggle to generate genuinely novel ideas without upstream human records.
- All traces, scratchpads, and reasoning streams are publicly released, including 41 curated full agent trajectories with tool calls and subagent behavior.
- Prime Intellect plans to extend speedruns to more of the training stack and use multi-agent harnesses with smaller open models to reduce compute costs.
Prime Intellect has published what it calls the largest open experiment on autonomous AI research: 153 autonomous runs across 18 frontier models, all competing on the same well-defined ML optimization task. The goal was not to crown a winner, but to measure how capable today's frontier models actually are at doing research autonomously, and to understand where they succeed and where they fall apart.
The arena: a constrained, measurable research task
The benchmark is the nanoGPT optimizer speedrun, a public leaderboard where the objective is to train a 124M-parameter GPT model to a fixed validation loss in as few steps as possible. The goal is simple: lower the number of steps needed to reach a target validation loss while only changing the optimizer, schedules, initialization, and some hyperparameters. No internet access. No changing the architecture. Just optimizer research, sandboxed on 8xH200s for up to 8 days per run.
This constraint is what makes it a meaningful proxy for research ability. The search space is large enough to require genuine hypothesis generation, but the feedback loop is tight enough to measure progress objectively. A human community built the current record over months of collaborative work, giving a concrete ceiling to measure against.
What they tested, and how
Prime Intellect ran every model inside their Prime Agent harness, which gives each model a persistent IPython kernel rather than a flat tool menu. Prime Agent treats context as a variable with programmatic access through a persistent IPython REPL, enables agent-to-agent communication and persistent sub-agents, and implements self-improvement through trajectory-based refinement. This means variables and state survive across turns, and models can build their own tooling mid-run rather than being constrained to pre-defined actions.