Artificial Analysis Patches Intelligence Index to Crown Claude Opus 5
Artificial Analysis patches its Intelligence Index to v4.1.1, upgrading grader models and fixing scoring errors in the τ³-Banking benchmark
Takeaways
- Artificial Analysis patched its Intelligence Index to v4.1.1, fixing grading errors without changing benchmark weights or rankings.
- τ³-Banking updated to v1.0.1, correcting scoring bugs in agent trajectories that recover from errors on multi-step banking tasks.
- HLE, AA-LCR, and AA-Omniscience now all use GPT-5.6 Luna as grader, replacing three different legacy models for consistency.
- Claude Opus 5 remains #1 with an Intelligence Index score of 63; most models move less than 1 point.
- Muse Spark 1.2 (xhigh) saw the largest gain at +2.7 points due to improved grading robustness.
- All scores on artificialanalysis.ai now reflect v4.1.1 automatically; free to access.
Artificial Analysis has patched its Intelligence Index from v4.1 to v4.1.1. The scope is narrow by design: no new benchmarks, no reshuffled weights. The fixes target grading errors and grader inconsistency across evaluations. Claude Opus 5 holds the top spot with an Index score of 63.
What changed
Two categories of fixes landed in this patch. The τ³-Banking benchmark was updated to its v1.0.1 release from Sierra Platform, and three evaluations received new grader models.
The v1.0.1 grading update corrects errors in the banking_knowledge task, specifically in trajectories where an agent recovers from a mistake mid-task. These "unhappy paths" were previously scored incorrectly. Scores from earlier versions are not directly comparable to v1.0.1 results.
τ³-Banking is a fintech customer-support benchmark from the τ-Knowledge framework. It tests whether agents can navigate a large unstructured knowledge base and execute multi-step tool calls to resolve realistic banking workflows. The corpus spans roughly 700 interconnected policy documents, around 195K tokens across 21 product categories. Accurate grading on recovery paths matters because real-world agents routinely hit dead ends before completing a task.
One grader for three evaluations
HLE, AA-LCR, and AA-Omniscience were previously graded by three different models: GPT-4o, Qwen3 235B A22B 2507, and Gemini 3 Flash Preview. All three now use GPT-5.6 Luna, a fast, cost-efficient model in OpenAI's GPT-5.6 series built for high-volume, latency-sensitive tasks. Artificial Analysis selected it based on strong agreement with human judgment in their grader validation process.
Different judges carry different calibration, which made cross-evaluation score comparisons unreliable. A single modern grader removes that inconsistency.
Score impact: small but real
Most models move less than one point on the Intelligence Index. The largest single-model swing is Muse Spark 1.2 (xhigh), which gains 2.7 points. The top of the leaderboard is unchanged.
- Claude Opus 5 — still #1 at an Index score of 63
- Muse Spark 1.2 (xhigh) — largest gain at +2.7 points
- Most other models — move less than 1 point
Artificial Analysis estimates a 95% confidence interval for the Intelligence Index of less than ±1%, based on experiments with more than 10 repeats across all included evaluation datasets. A sub-point shift from a grader upgrade sits well within that margin, which is the point: the patch tightens reliability without distorting rankings.
Where to find updated scores
All published scores on artificialanalysis.ai now reflect v4.1.1. The methodology page covers each evaluation and its weight in the Index. If you use Artificial Analysis scores for model selection, no action is needed — the site updated automatically and the changelog documents every change. The index is free to browse.