Skip to main content
Back

IBM Research's PROVE Trains AI Agents to Chain 343 Tools Without Breaking

IBM Research's PROVE framework trains small LLMs to reliably chain multi-step tool calls using live MCP servers and a judge-free reward system, beating larger models on key benchmarks.

Paper, Full Access
  • IBM Research releases PROVE, a full RL training framework for multi-step tool use in live MCP environments.
  • Framework includes 20 stateful MCP servers exposing 343 tools, with session-scoped isolation for safe parallel RL training.
  • A dependency-graph-guided data synthesis pipeline ensures every training query references entities that actually exist in the server state.
  • A five-component programmatic reward (no judge model) penalizes verbose tool calling and rewards correct argument values and tool ordering.
  • Four sub-10B models (Qwen3-4B/8B, Qwen2.5-7B, Granite-4.1-8B) trained on ~13K examples gain up to +10.2 on BFCL Multi-Turn, +6.8 on tau2-bench, +6.5 on T-Eval.
  • Code and models not yet publicly released; full methodology available in the preprint.

Getting a language model to call one tool correctly is a solved problem. Getting it to call ten tools in the right order, where each call depends on the result of the last, against a live system that actually changes state -- that is where most agent pipelines quietly break down. IBM Research just published PROVE (Programmatic Rewards On Verified Environments), a framework that tackles this problem end-to-end: it builds the training environments, synthesizes grounded data, and defines a reward signal -- all without a judge model.

Why multi-step tool use is still broken

The field has made enormous progress on single-turn function calling. But multi-step orchestration -- where a model must book a flight, then look up the confirmation, then update a calendar entry -- requires something qualitatively different. Training LLMs to orchestrate multi-step tool calls is held back by three coupled obstacles: realistic stateful execution environments are costly to build, synthetic training queries are often detached from the server's actual state, and recall-based RL rewards incentivize verbose tool-calling patterns.

That last point is subtle but important. If you reward a model for how many correct tools it mentions (recall), it learns to call everything just to be safe -- a kind of tool-calling verbosity that looks good on training metrics but is useless in production where every extra call costs latency and money.

The data problem is equally nasty. Most synthetic training pipelines generate queries in isolation, without checking whether the entities they reference actually exist in the server's current state. A query like "cancel my reservation with ID 4821" fails immediately if that reservation was never created. RLVR (Reinforcement Learning with Verifiable Rewards) trains models on tasks where correctness can be automatically checked. Instead of human preference labels, the reward signal comes from programmatic verification. But applying this to tool use requires environments that can actually execute and verify those tool calls -- which is exactly what has been missing.