Most AI benchmarks answer a narrow question: can this model solve a math competition problem, or pass a bar exam? What they rarely answer is the question that actually matters for deployment: which model should I use for the work my team does every day? Artificial Analysis just took a direct swing at that gap with six new Capability Indices covering Finance & Accounting, Legal, Healthcare & Medical, Strategy & Ops, Engineering, and Economics.
The problem with generic benchmarks
The AI benchmarking landscape has a saturation problem. MMLU and MMLU-Pro are functionally saturated above 88% for frontier models, making score differences at the top statistically meaningless. The field responded by building harder tests, but even that misses the point. Benchmarks may not always map to real-world results. As one researcher put it: "Knowing that a benchmark for legal reasoning has 75% accuracy tells us little about how well it would fit in a law practice's activities."
Artificial Analysis has been pushing toward economically grounded evaluation for a while. The emphasis on economically measurable output is a philosophical shift in how the industry thinks about AI capability. Rather than asking whether a model can pass a bar exam or solve competition math problems, the new benchmarks ask whether AI can actually do jobs. The six new industry indices are the most concrete expression of that philosophy yet.
How the indices are built
Each index is a weighted composite of existing benchmarks, but the weighting logic is what makes it different. Rather than equal-weighting a grab-bag of evals, Artificial Analysis maps real job tasks using
Don't miss what's next in AI
Join 300,000+ engineers and researchers who get the signal, not the noise.
- Full access to in-depth AI research breakdowns
- Be the first to know what's trending before it hits mainstream
- Daily curated papers, repos, and industry moves