Epoch's EBR-Bench Reveals Humans Learn While Frontier AI Stays Stuck

Epoch AI has published human baseline results for EBR-bench, a benchmark that measures whether AI systems can learn a complex, unfamiliar task through repeated attempts. The headline result is blunt: humans get better at the game with practice, and frontier models basically do not.

EBR-bench uses Earthborne Rangers, a campaign-style board game in which a player explores a wilderness, manages a card deck, and pursues objectives across in-game days. Epoch chose it for three reasons: it is relatively obscure, so models are unlikely to have memorized it during training; it is almost entirely card-based with very little spatial reasoning; and it demands a layered mix of strategy and tactics over a long time horizon. One playthrough of the evaluated segment takes an experienced human two to four hours, and full mastery can require dozens of runs.

Rigging the test for a fair fight

The evaluation is designed so learning is the whole point. Each model gets 10 playthroughs by default. The agent is told only the final 2 will be scored, leaving the earlier ones free for exploration and note-taking. Notes are the agent's only mechanism for carrying knowledge across context compactions, and the topline score is the best of those last two runs.

Under the hood, models act through a simple ReAct agent harness with a small tool set: a card lookup utility, a map image, a text pathfinding tool for models that struggle with vision, and a notes directory. Each model is also run at two compaction threshold settings, 250,000 tokens and a higher value around 90% of the model's reported maximum context window, with the better of the two topline scores reported.

The human baseline, matched to the machines

To find out what "good" actually looks like, Epoch recruited 15 human players and put them through nearly the same pipeline as the models. Participants came from the personal and professional networks of the benchmark creators, and were compensated at $30/hr with a cap of 60 hours. A +25% bonus was offered for a topline score of 10 or above ("better than AI"), and an additional +25% bonus for a topline score of 15 or above ("expert performance").

Participants played through a private website built on the same game engine provided to the agents, with light UI affordances such as boxed card information and color-coded keywords. Both groups received the exact same information about the game, and humans were strongly discouraged from consulting the internet. Of the 15 initial recruits, 2 dropped out early and were excluded from the published dataset. Aggregate statistics are reported for the 13 remaining players, alongside the data of the top human, who hit the maximum score of 21/21 on their 5th playthrough.

People start rough and end up dominant

Most participants had never played Earthborne Rangers, including the top scorer. Even so, roughly half of the humans outscored every AI system Epoch has evaluated, and almost all showed at least some improvement across their runs. Several frontier agents actually beat most humans on the very first playthrough, but that lead evaporated as the humans learned.

Two participants reached a perfect 21 out of 21, and the efficiency gap between them is worth noting. One hit the ceiling after 61 hours across all playthroughs; the other got there in 32. More practice time generally translated to higher scores, but with substantial variance, suggesting the game rewards deliberate thinking rather than hours logged.

Where AI actually breaks down

The tactical picture is where the story gets uncomfortable for scaling optimists. Earthborne Rangers uses a mechanic called "fatigue" as its main damage system. Incoming fatigue moves cards from the top of the player's deck to their fatigue stack, and an in-game day ends when the deck runs out of cards. Health, remaining time, and card resources are therefore all represented at once by the cards left in the deck, and managing that pool is the core skill of the game.

An expert human player can keep average fatigue per round to about 0.6, which corresponds to roughly 15 rounds of gameplay per day before their deck runs out. Completely random play incurs 3.5 fatigue per round, or only about 5.3 playable rounds per day. Frontier models like GPT-5.5 and Claude Opus 4.8 land closer to the random baseline, incurring around 2.1 fatigue per round for roughly 7.7 rounds of play per day. Humans start below this line and typically surpass the agents within a handful of playthroughs.

Strategic exploration tells a similar story. Before each playthrough, players build a 24-card deck, and deck power varies drastically, with the maximum score of the strongest deck potentially twice that of the weakest. Coarsely counting the combinations of background, specialty, and role cards yields 32 deck archetypes. Humans sample noticeably more of them; models tend to lock in on a favorite and grind it.

Handing over the answer key barely helps

Epoch also ran a "max elicitation" condition where agents receive a detailed strategy guide written by an expert player. Compared to the ten-playthrough basic setup, models achieve between 2 and 3.5 additional objectives out of 21, depending on the model, which the authors characterize as a modest effect given that the setting essentially hands over the answer sheet. Ad hoc experiments with code execution, Claude Code, and Codex as scaffolds also failed to produce significant gains.

Why a board game bench matters

EBR-bench is a proxy for on-the-fly learning, the ability to walk into an unfamiliar task and get materially better at it through practice. If that ability existed at scale in current models, we would expect to see it here.

The Epoch team lays out a few takeaways worth internalizing:

• Inference compute scaling does not overcome every capability deficiency, and the returns to repeated play for an out-of-distribution game appear minimal.
• Even with a concrete score to optimize against, current agents do not reliably climb the gradient through experience alone.
• Since focused training could plausibly improve performance on EBR and similar games, the current results are interpreted as evidence that AI systems remain fairly limited by their training distributions.
• Note-taking as a substitute for weight updates has real limits; even good externally supplied notes only close part of the gap.

For anyone building agentic systems, the practical implication is that "just let it try again" is not yet a substitute for fine-tuning on the target task or providing highly structured scaffolds. Repeated attempts help humans a lot and models very little, and any pipeline that assumes otherwise is banking on a capability that has not yet shown up on this benchmark.