Artificial Analysis Launches Progress Index to Track AI Gains Across Years

Artificial Analysis has launched the Progress Index v1, a longitudinal benchmark that maps its historical language-model results onto a shared capability scale. The index allows researchers and developers to compare frontier models across years, even as individual benchmarks are added or retired.

Benchmark suites lose value once leading models saturate them. Artificial Analysis addresses that problem by separating two jobs: The Intelligence Index is designed to measure current frontier capability with evaluations that still differentiate leading models, while the Progress Index tracks capability changes over time. Scores from different Intelligence Index versions remain unsuitable for historical comparisons.

One scale for changing benchmarks

The Progress Index uses item response theory, or IRT, the statistical framework behind standardized tests such as the SAT. IRT jointly estimates each model’s capability and each task’s difficulty, placing both on a common scale.

Models do not need results from identical benchmark sets. Each task’s difficulty is estimated from the models that attempted it, while each model’s capability is estimated from its completed tasks. Overlapping evaluations connect the data, allowing models tested on older MMLU-era benchmarks to share an axis with models evaluated on newer suites such as AA-Briefcase and Terminal-Bench.

Fit rules at a glance

• The scale anchors o3-mini (high) at 220 and GPT-5.5 (xhigh) at 320. A fixed point gap represents the same multiplicative change in success odds anywhere on the scale.
• A benchmark requires results from at least four models, and a model requires results on at least four benchmarks, to enter the fit.
• Benchmark results are normalized to a 0-to-1 range before fitting, with chance performance rescaled to zero. A 25% score on four-option GPQA Diamond therefore counts as zero.
• Each benchmark receives equal weight for a given model, regardless of how many tasks it contains.

Artificial Analysis cites the Epoch AI Capabilities Index and Anthropic’s capability scale from the Claude Mythos Preview system card as precedents. Its implementation fits individual tasks with log loss and uses only evaluations run by Artificial Analysis.

Frontier scores are rising faster

Reasoning-focused releases have pushed annual frontier-score gains well above the 2023 increase. The number of release dates that established a new record has also climbed sharply.

New frontier records have arrived about once every four weeks during 2026. OpenAI held the highest score continuously for the first 27 months in the dataset, while the lead has since moved repeatedly among OpenAI, Anthropic, and Google.

A 50-fold price drop in five months

Output-token prices have fallen quickly within comparable capability bands. GPT-5.3 Codex, the first model to exceed a Progress Index score of 300, cost $14 per million output tokens in February 2026. DeepSeek V4 Flash 0731 reached the same tier in July at $0.28 per million, reducing the listed output price by a factor of 50.

Deployment costs also depend on input pricing, latency, context limits, tool support, reliability, and workload-specific accuracy. The cost curve isolates output-token pricing at similar composite capability rather than total application cost.

Open weights trail by four months

Open-weight models reach a given capability level a median of about four months after proprietary models. DeepSeek V4.1 Flash 1017 currently leads released-weight models with a score of 324, matching the level Claude Fable 5 reached in June 2026. Kimi K3.1 Pro scores 341, about two months behind the closed frontier, although its weights have yet to be released.

China holds the open-weight lead

Chinese releases have recorded the highest open-weight score since September 2025. DeepSeek V4.1 Flash 1017, at 324, leads the top US open-weight entry, Muse Spark Open, at 307. The United States and China exchanged the lead seven times between July 2023 and September 2025.

Choose the index for the job

Composite scores can narrow a model shortlist, while deployment decisions still require evaluations built around the target workload. Artificial Analysis publishes interactive country comparisons, frontier histories, and cost-per-capability curves in its interactive Trends charts.