Artificial Analysis Rebuilds Intelligence Index v4.1 Around Agentic AI Tasks

Benchmarks have a shelf life. Once frontier models start clustering at the top, the scores stop telling you anything useful. Artificial Analysis just released Intelligence Index v4.1, a significant overhaul of its composite AI scoring system, and the headline change is a deliberate pivot: the index now weights agentic tasks more heavily than any other category, reflecting where the real capability race is happening in 2026.

What actually changed

Three core updates define v4.1. First, the benchmark composition itself was restructured. Terminal-Bench Hard was upgraded to Terminal-Bench 2.1 and τ²-Bench Telecom was replaced by τ³-Bench Banking, both moving to newer, more robust task sets with harder, more realistic agentic scenarios that better separate frontier models. The Intelligence Index is now calculated as a weighted average across four categories: Agents (34%), Coding (24%), Scientific Reasoning (24%), and General (18%), with the weighting explicitly emphasizing agentic tasks.

Second, GDPval-AA, the highest-weighted single evaluation in the index at 20%, got a meaningful upgrade. GDPval-AA v2 re-baselines Elo scores to human expert performance at 1000, introduces a panel of three frontier LLM judges from leading labs replacing a single judge, and expands turn limits to 250 to allow for even longer-horizon agent trajectories. This benchmark tests models on real knowledge-work deliverables across 44 occupations, graded by blind pairwise comparison.

Third, IFBench was dropped from the index entirely. AI benchmarks usually have a short shelf life: once models start scoring near the top, the evals stop being useful for telling systems apart, and most evaluations added to the Intelligence Index saturate within about six months. IFBench, which tested precise instruction-following on novel constraints, finally hit that wall with frontier models. Artificial Analysis says it will continue running and publishing IFBench results, just not counting them in the composite score.

The full benchmark lineup

Intelligence Index v4.1 incorporates 9 evaluations: GDPval-AA v2, τ³-Banking, Terminal-Bench v2.1, SciCode, Humanity's Last Exam, GPQA Diamond, CritPt, AA-Omniscience, and AA-LCR. Here are the weights:

• GDPval-AA v2 (20%) , real-world knowledge work across 44 occupations, scored by Elo
• Terminal-Bench 2.1 (16%) , agentic terminal tasks: compiling code, configuring servers, training models
• τ³-Bench Banking (14%) , multi-turn, tool-using customer-support scenarios in a simulated retail banking environment
• Humanity's Last Exam (12%) , expert-level academic reasoning across math, science, humanities
• AA-Omniscience Accuracy (8%) , factual knowledge with hallucination penalty
• SciCode (8%) , Python code for scientific computing tasks
• GPQA Diamond (6%) , graduate-level biology, physics, and chemistry
• AA-LCR (6%) , long-context reasoning over ~100k-token documents
• CritPt (6%) , research-level physics problems
• AA-Omniscience Non-Hallucination (4%) , penalizes confident wrong answers

Why agentic benchmarks are getting harder to game

Terminal-Bench was chosen partly because it is quite challenging, with no model scoring above 50% on the hard tasks upon its initial release. The benchmark comprises hand-crafted, human-verified tasks spanning scientific computing, software engineering, machine learning, security, and system administration , and unlike traditional code generation benchmarks that evaluate isolated functions, Terminal-Bench tasks require agents to complete end-to-end workflows. The τ³-Bench Banking upgrade follows the same logic: move to a domain where models can't just pattern-match on training data.

The new per-task metrics are arguably the most practically useful addition. Artificial Analysis now reports Cost per Task, Time per Task, and Tokens per Task for every model, calculated by dividing the total resources consumed running the full Intelligence Index by the number of tasks across all evaluations. This gives you a normalized, apples-to-apples view of what each model actually costs to operate on agentic workloads.

The numbers that matter

The leaderboard results reveal a striking split between proprietary and open-weight models. On the intelligence side, the gap is real but not enormous. Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) currently has the highest score, but is unavailable. Among available models, Claude Opus 4.8 (max) leads at 56, followed closely by GPT-5.5 (xhigh) at 55.

On cost, the gap is enormous. DeepSeek V4 Pro (max) completes a single Intelligence Index task for $0.04. Claude Opus 4.8 (max) costs $1.78 per task, and Claude Fable 5 tops out at $3.25. That's a 45x cost difference for a model that scores 12 points lower on the index. The top open-weight models by Intelligence Index are MiniMax-M3 (44), DeepSeek V4 Pro (Reasoning, Max Effort) (44), and Kimi K2.6 (43).

Time per task tells a different story. Intelligence Index is calculated as a weighted average across four categories. Inference time ranges from 1.5 minutes for Grok 4.3 (high) to 13.5 minutes for Claude Sonnet 4.6 (max), a roughly 9x spread. Gemini 3.1 Pro Preview stands out on the time-vs-intelligence chart, completing tasks in 1.6 minutes for an index score of 46.

The cost-efficiency picture for open-weight models

DeepSeek V4 Pro is a 1.6-trillion parameter Mixture-of-Experts model released under the MIT license, designed for advanced reasoning, complex software engineering, and long-running agentic tasks. However, V4 Pro has a 94% hallucination rate on the AA-Omniscience benchmark, meaning when the model does not know an answer, it nearly always responds anyway rather than abstaining , a real liability for production agentic systems where confident wrong answers can cascade into downstream failures.

The open-weight field is genuinely competitive now. V4 is the first new architecture from DeepSeek since V3, introducing V4 Pro at 1.6T total / 49B active parameters and V4 Flash at 284B total / 13B active parameters , DeepSeek's first two-tier lineup, with Pro positioned for maximum capability and Flash for faster, lower-cost inference. MiniMax M3 ties V4 Pro at 44 on the index, while Kimi K2.6 trails by one point at 43.

What this index revision signals

The shift to agentic-heavy weighting isn't just a methodological update , it's a statement about where model capability actually matters. Static question-answering benchmarks are increasingly poor proxies for what models do in production: multi-step tool use, long-horizon planning, real file outputs, and recovery from errors. AI benchmarks usually have a short shelf life, and once models start scoring near the top, the evals stop being useful for telling systems apart. By retiring saturated benchmarks and replacing them with harder agentic tasks, Artificial Analysis is trying to stay ahead of that curve.

The cached input token reporting is also worth noting for teams running models at scale. Artificial Analysis now reports cached input tokens and their impact on cost, which can meaningfully change the economics of models that charge different rates for cache hits versus cold inputs. For long-running agents that repeatedly reference the same context, this can flip the cost ranking between models.

The full methodology, including every prompt template, scoring function, and evaluation weight, is publicly documented. The live leaderboard is available now with per-model breakdowns across all 9 evaluations, plus the new cost, time, and token-per-task metrics.