Skip to main content
Back

Artificial Analysis Releases 6 Industry Indices That Reshape How Developers Pick AI Models

Artificial Analysis launches six domain-specific AI benchmarks grounded in real job tasks, revealing sharp tradeoffs between frontier performance and cost.

  • Artificial Analysis launched six domain-specific AI benchmark indices: Finance & Accounting, Legal, Healthcare & Medical, Strategy & Ops, Engineering, and Economics.
  • Each index is built from O*NET job task data, with benchmarks weighted by how often each capability appears in real professional work.
  • Claude Fable 5 leads all eight Capability Indices; GLM-5.2 (open-weights) leads five of six industry indices, scoring within 2 points of Claude Sonnet 5 on Engineering.
  • Cost tradeoffs are steep: Claude Fable 5 costs over 100x more per task than DeepSeek V4 Pro on Strategy & Ops, for a 12-point score advantage.
  • Time per task spans 15x within each index; Gemini 3.1 Pro Preview completes Legal tasks 7x faster than Claude Fable 5 while scoring only 11 points lower.
  • All indices, weightings, and methodology are publicly documented and free to explore at artificialanalysis.ai.

Most AI benchmarks answer a narrow question: can this model solve a math competition problem, or pass a bar exam? What they rarely answer is the question that actually matters for deployment: which model should I use for the work my team does every day? Artificial Analysis just took a direct swing at that gap with six new Capability Indices covering Finance & Accounting, Legal, Healthcare & Medical, Strategy & Ops, Engineering, and Economics.

The problem with generic benchmarks

The AI benchmarking landscape has a saturation problem. MMLU and MMLU-Pro are functionally saturated above 88% for frontier models, making score differences at the top statistically meaningless. The field responded by building harder tests, but even that misses the point. Benchmarks may not always map to real-world results. As one researcher put it: "Knowing that a benchmark for legal reasoning has 75% accuracy tells us little about how well it would fit in a law practice's activities."

Artificial Analysis has been pushing toward economically grounded evaluation for a while. The emphasis on economically measurable output is a philosophical shift in how the industry thinks about AI capability. Rather than asking whether a model can pass a bar exam or solve competition math problems, the new benchmarks ask whether AI can actually do jobs. The six new industry indices are the most concrete expression of that philosophy yet.

How the indices are built

Each index is a weighted composite of existing benchmarks, but the weighting logic is what makes it different. Rather than equal-weighting a grab-bag of evals, Artificial Analysis maps real job tasks using O*NET (a U.S. Department of Labor taxonomy of occupational activities), then selects benchmarks that best represent those tasks and weights them by how frequently each capability appears in that profession.

For example, the Legal Index weights legal knowledge most heavily (35%), followed by agentic knowledge work (25%), multi-step reasoning (15%), long-context reading (10%), and non-hallucination (10%). The Healthcare index similarly prioritizes clinical knowledge (35%) and penalizes hallucination with a dedicated 15% weight, since fabricated drug interactions or dosing guidelines are a direct patient safety risk.

The underlying benchmarks include:

  • AA-Omniscience - Artificial Analysis's own knowledge and hallucination benchmark across 6,000 questions in domains like law, health, business, and engineering
  • GDPval-AA v2 - A private evaluation for frontier agentic capability in long-horizon knowledge work, testing agents on realistic business workflows that require deliverables such as spreadsheets, presentations, and memos
  • Humanity's Last Exam (HLE) - A frontier-level benchmark with 2,500 expert-vetted questions across mathematics, sciences, and humanities
  • GPQA Diamond - The most challenging 198 questions from GPQA, where PhD experts achieve 65% accuracy but skilled non-experts only reach 34% despite web access
  • Terminal-Bench v2.1 - Tests models operating real terminal environments for builds, scripts, and debugging
  • AA-LCR - Long-context reading and reasoning across extended documents

Every component benchmark is run independently by Artificial Analysis, not sourced from model providers' self-reported numbers. The full methodology and per-index weightings are publicly documented.

Who's on top, and where rankings diverge

Claude Fable 5 (with Opus 4.8 fallback) leads all eight Capability Indices, with Claude Opus 4.8 (max) in second place on six of the eight. But the more interesting story is what happens below the top two: rankings reshuffle substantially by domain between Gemini 3.5 Flash, Gemini 3.1 Pro Preview, GPT-5.5 (xhigh), Claude Sonnet 5 (max), and GLM-5.2 (max). A model that ranks third overall might rank sixth in legal and second in engineering. That's exactly the kind of signal a generic leaderboard can't give you.

For open-weights models specifically, the results are striking. GLM-5.2 (max) leads on five of the six industry indices. On the Engineering Index, it scores 53, within 2 points of Claude Sonnet 5 (max) at 55 and GPT-5.5 (xhigh) also at 55. DeepSeek V4 Pro (max) takes the open-weights lead on the Strategy & Ops Index.

The cost and speed tradeoffs are brutal

The indices don't just report accuracy scores. They also track Cost per Task and Time per Task, which is where the data gets genuinely useful for deployment decisions.

  • DeepSeek V4 Flash (max) completes tasks for under $0.04 across all six indices while scoring mid-pack
  • GLM-5.2 (max) leads open-weights performance at $0.26 to $0.58 per task
  • Claude Fable 5 on the Strategy & Ops Index costs $3.48 per task, over 100x more than DeepSeek V4 Pro (max) at $0.03, for a 12-point score advantage
  • Time per task spans roughly 15x within each index, from 1.1 minutes (Nova 2.0 Pro Preview) to 16.7 minutes (Claude Sonnet 5 max)
  • On the Legal Index, Gemini 3.1 Pro Preview completes tasks in 0.8 minutes vs. 5.4 minutes for Claude Fable 5, while scoring only 11 points lower (48 vs. 59)

That last point is worth sitting with. An 11-point gap in legal benchmark score costs you 6.6x more time per task. Whether that tradeoff is worth it depends entirely on your use case and volume.

What this changes about how you should pick a model

The practical implication is that model selection should now be domain-first. If you're building a legal research tool, the Engineering Index score is irrelevant. If you're automating financial modeling workflows, the Healthcare non-hallucination weight doesn't apply to you. These indices let you filter to the benchmarks that actually reflect your workload.

The indices also surface a real tension in the open vs. closed model debate. As of early 2026, Anthropic, xAI, Google, OpenAI, Alibaba, and DeepSeek all occupy the top tier of general performance ratings, shifting competitive pressure toward cost, reliability, and domain-specific performance. GLM-5.2's strong showing across five of six industry indices suggests that for many professional domains, open-weights models are now within striking distance of frontier closed models, at a fraction of the cost.

The six new indices are live now on the Artificial Analysis website, free to explore, with full score breakdowns, cost curves, and speed data for all evaluated models.