Sci-PRM Teaches AI to Verify Scientific Reasoning With Real Tool Calls
Sci-PRM brings step-by-step reasoning verification to biology, chemistry, and physics by teaching reward models to evaluate tool use, not just logic
Takeaways
- Sci-PRM is a new process reward model that verifies scientific reasoning chains involving tool calls, covering biology, chemistry, and physics.
- SCIPRM70K is a 70,000-trajectory dataset of Chain-of-Tool reasoning, where tool invocations are explicitly interleaved with reasoning steps and labeled for correctness.
- The model scores each step across three axes: tool selection, tool execution accuracy, and result interpretation -- all in a single forward pass.
- Sci-PRM enables Best-of-N selection at inference time and acts as a dense reward signal during RL training, directly addressing the "advantage disappearance" problem in sparse-reward settings.
- Benchmarks tested include MSEarth, BioProBench, Mol-Instruct, and ChemBench, with Sci-PRM outperforming outcome-reward baselines on all four.
- No public code or model weights are released with the paper at this time; the model's reliability is also limited to tools within its training distribution.
Process Reward Models (PRMs) -- the judges that score each step of a model's reasoning chain -- have become a cornerstone of modern AI performance in math. But science is not math. When a model reasons through a chemistry problem, it needs to call a molecular simulation tool, interpret the output, and carry that result forward without hallucinating. No existing PRM was built to verify that. A new paper introduces Sci-PRM, a reward model specifically designed to evaluate scientific reasoning that weaves tool calls into the chain of thought.
Why math PRMs break on science
A PRM, in plain terms, is a model trained to look at a partially completed reasoning chain and say "this step is correct" or "this step is wrong." PRMs estimate the quality of each partial reasoning state by assigning a reward that reflects the likelihood of the chain eventually leading to a correct solution. They have proven powerful for math, where correctness is deterministic and verifiable. But scientific domains like biology, chemistry, and physics introduce a fundamentally different challenge.
Scientific reasoning requires not just logical steps, but also:
- Selecting the right domain-specific tool (a SMILES parser, a protein structure predictor, a physics simulator)
- Executing that tool correctly with the right inputs
- Interpreting the tool's output accurately before continuing to reason
Current models frequently hallucinate tool outputs or misinterpret results. And because existing PRMs were trained only on mathematical reasoning trajectories, they have no mechanism to verify whether a tool was called correctly, or whether the result was used faithfully. PRM provides a fine-grained evaluation for each reasoning step and has shown better results than outcome-based models, but data for PRM is extremely scarce and annotation is costly. In science, that annotation problem is compounded by the need for domain expertise across multiple fields simultaneously.
Building the dataset from scratch
The team's first contribution is SCIPRM70K, a large-scale dataset of 70,000 scientific reasoning trajectories. Each trajectory is a "Chain-of-Tool" sequence -- a reasoning chain where tool calls are explicitly interleaved with the model's thinking steps. The pipeline to build it has four stages: