Vals Science Index reshuffles the frontier-model leaderboard

Vals AI has launched the Science Index, an equal-weight leaderboard for models operating as scientific agents. Its three benchmarks give each system a biological dataset, an experimental environment, or a terminal, then measure its ability to investigate data, test hypotheses, and complete research workflows.

General-purpose leaderboards blend coding, reasoning, and knowledge tasks. The Science Index produces a different order: GPT-6 Astra ranks seventh on the general Vals leaderboard and first here, while Gemini 4 Argon moves from first overall to fifth in science. For developers building research agents, that reshuffling affects model selection, operating cost, and latency.

Astra leads a reordered field

GPT-6 Astra tops the index at 65.09%, leading GPT-6.1 Sol by 5.46 percentage points. Sol and Claude Sonnet 5.5 are separated by 0.51 points, yet Sol costs $1.36 per test compared with Sonnet’s $13.24 and completes the test in less than half the time. Gemini 4 Argon has the highest reported cost among the top five at $18.40 per test.

Three tests feed one score

The index takes the arithmetic mean of three benchmark scores, giving each benchmark one-third of the final result. The tests cover distinct parts of scientific work, from deriving hidden equations to analyzing biological data and completing shell-based research tasks.

• MysteryMechanism, built by Vals: Agents investigate 222 hidden mechanisms drawn from biology, physics, chemistry, engineering, ecology, and abstract dynamics. Names and domain hints are removed. Each agent receives two observations collected without intervention, a budget of 2d + 1 experiments, where d represents task dimensionality, and must submit an executable formula.
• BioMysteryBench, built by Anthropic: Agents conduct open-ended investigations of real biological datasets to answer unresolved questions. The benchmark authors designate a subset of tasks as beyond human-expert solvability, leaving room to measure performance above human baselines.
• Terminal-Bench-Science, built by Steven Dillmann and the Terminal-Bench team: Agents complete research workflows from a shell across life, physical, earth, mathematical, and engineering sciences.

The average conceals specialists

Domain-level results divide the lead among three models. GPT-6 Astra ranks first in life and mathematical sciences, Claude Sonnet 5.5 leads physical and engineering sciences, and GPT-6.1 Sol leads earth sciences. Sonnet ranks third overall and 11th in earth science, illustrating how the composite score can obscure weaknesses relevant to a specific deployment.

Benchmark-level wins also split across vendors. Claude Sonnet 5.5 leads BioMysteryBench, while GPT-6 Astra leads MysteryMechanism and Terminal-Bench-Science. A bioinformatics agent, laboratory automation system, and cross-discipline terminal agent can therefore produce different model shortlists from the same overall table.

Anonymous tasks reveal agent strategy

MysteryMechanism reduces direct recall opportunities by removing mechanism names and domain labels. Vals reports that roughly 90% of Astra’s successful traces never identified the underlying domain. The trace pattern supports mechanism inference as the source of many successes; trace data alone cannot exclude the use of familiar structural patterns.

Vals also observed different experimental strategies. Astra tended to run experiments sequentially and eliminate competing hypotheses, whereas Claude Opus 5.5 often committed to a candidate formula earlier. Those behaviors can affect reliability when an agent faces noisy data, limited experiment budgets, or expensive laboratory actions.

Reasoning-budget sweeps add an operational constraint. According to Vals, maximum reasoning effort can increase cost by as much as 20 times after accuracy has plateaued at a model-specific setting. Developers can control that expense by testing several reasoning levels per workflow and selecting the lowest setting that preserves accuracy.

Choose by workload and verify locally

Scientific-agent deployments compress several constraints into one score: discipline, task type, budget, latency, and tool configuration. The MysteryMechanism results and BioMysteryBench results provide model-level cost, latency, and uncertainty estimates for a closer comparison.

• Match the task: Select the benchmark and domain closest to the intended research workflow.
• Model operating cost: Estimate spend at the expected test volume and reasoning setting, including anticipated retries.
• Reproduce the configuration: Record the model version, agent scaffold, prompts, tool permissions, time limits, and inference settings.
• Run a local holdout: Evaluate representative data and tasks that were unavailable during prompt and agent development.

Vals says it plans to add further scientific benchmarks over time. Because benchmark composition affects an equal-weight average, teams tracking progress should record the index date, benchmark versions, model version, and inference settings. The current results show that workload-specific evaluation can change both the selected model and the cost of operating it.