Agent Arena Splits Into 3 Categories, Proving No Single Model Wins

The Agent Arena leaderboard just got a lot more useful for picking a model. Instead of a single ranking that mashes every kind of agent task together, the board now splits into three categories, adds real per-task dollar costs, and includes a Pareto view showing which models are actually cost-efficient at each performance tier. The headline finding: no single model wins across the board.

The top-ranked model varies by category. GPT 5.6 Sol (xHigh) leads Code, Claude Opus 5 (High) leads Work, and Claude Opus 5 (Max) leads Chat. That split matters because the underlying data comes from real usage, not a synthetic benchmark, on the Agent Arena leaderboard covering 49 models and roughly 1.8 million sessions.

Why one leaderboard was not enough

Agents are being asked to do wildly different things in production: refactor code, draft reports, run research, or just chat. A single averaged score hides the fact that a model can be great at bash recovery and bad at steerability, or cheap on short tasks and ruinously expensive on long ones. The update, described in the Arena team's writeup, is built around two decisions that pick the model.

• What's best for my kind of task? Solved by the new Code, Chat, and Work category filters.
• What will it cost to actually complete that task? Solved by the new task-based cost metric and Pareto frontier view.

Task-based cost, not per-token cost

Per-token pricing is misleading for agents because a single long agent session can cost hundreds or even a thousand dollars once you factor in context growth, tool calls, and caching. Arena instead segments Agent Mode sessions into discrete tasks (for example, a four-turn back-and-forth to rearrange sheet music counts as one task) and reports the median price to complete one. The board now shows three cost columns: Cost/Task (P50), Output Tokens/Task (P50), and the raw Price $/M for input and output.

The numbers surface some real trade-offs. Claude Opus 5 (High) tops the overall board with 12.19% net improvement at $1.84 per task and 24.6K output tokens. GPT 5.6 Sol (xHigh) sits close behind at $1.38 per task with 17.2K tokens. Further down, open-weight options like GLM 5.2 (Max) deliver ~6.7% net improvement at $0.17 per task, and Deepseek V4 Flash lands around $0.04 per task, which changes the calculus for high-volume workloads.

What the categories actually contain

Arena classified the 1.7M+ sessions and found Work dominates at 69.1% of tasks, with Chat (39.0%) and Code (38.8%) roughly tied. Tasks can belong to multiple categories, which is why the numbers sum above 100%. Example prompts from the release give a feel for the split:

• Code: Build a playable browser game with ray tracing, 60fps target, keyboard and touch controls, pause and restart states, and a local high-score table.
• Chat: Deep-search news, studies, and social media for new approaches to CFS/ME, plus ways to boost mental and physical energy.
• Work: Produce a full report on thermal insulation performance testing, including HVAC and greenhouse case studies, classifications, cost, and working principles.

The per-signal picture

Rankings are built from several behavioral signals rather than a single win rate. Agent Arena goes beyond human preference, measuring task success rates, textual edits, steerability, tool hallucination rates, and more. The current signal leaders show how uneven model strengths are:

• Confirmed Success (user says the task is done): Claude Opus 5 (Max) at 17.96%
• Praise vs Complaint: Claude Fable 5 (High) at 25.14%
• Steerability (landing corrections when users push back): Claude Opus 5 (High) at 11.34%
• Bash Recovery (fewest steps after a failed command): GPT 5.5 (xHigh) at 14.53%
• Tool Hallucination (invoking tools that don't exist): a cluster of frontier models tied at 1.22%

How to actually use it

The practical value is that you can now match a model to a workload with real data behind it. If you are wiring up a coding agent, filter to Code and check Bash Recovery and Steerability, then look at Cost/Task before committing. If you are running long-horizon research or report generation, Work is the relevant lane, and the Pareto view will show whether paying for a top-tier Opus is worth it over a mid-tier option that lands 80% of the quality for a fraction of the spend.

The board updates continuously from live sessions on Agent Mode, so rankings shift as new models arrive and usage patterns change. For teams evaluating agent stacks, that dynamic-with-cost view is closer to a procurement dashboard than a traditional benchmark, and it makes the old habit of picking the top of a single leaderboard look pretty naive.