EAPO Fixes AI Agents That Compulsively Call Tools, Boosting Accuracy by 10%
EAPO teaches RL-trained agents to skip unnecessary tool calls, boosting accuracy by up to 10% while cutting tool use by up to 25%
Takeaways
- Tool abuse problem: RL-trained agents overuse tools like web search and code interpreters even for simple queries solvable by internal reasoning.
- EAPO framework: Introduces tool-free trajectories, difficulty-aware reward penalties, and confidence-aware token reweighting to fix this during training.
- Key results: Up to +10.45% accuracy improvement over GRPO while reducing tool calls by up to 24.59% across three model families.
- Models tested: Qwen2.5-3B, Qwen2.5-7B, and Llama3.1-8B evaluated on nine math and knowledge-intensive benchmarks.
- Practical impact: Better accuracy AND fewer tool calls simultaneously, disproving the assumption that restricting tools hurts performance.
- Availability: No code or weights released yet; the paper provides sufficient detail for reimplementation on top of existing GRPO pipelines.
When you give a language model access to tools like a code interpreter or a web search engine, something strange happens during reinforcement learning training: the model starts calling those tools compulsively, even for questions it could easily answer from memory. Ask it what 1+1 equals and it might spin up a Python interpreter. Ask it which direction the sun rises and it fires off a web search. This behavior, called tool abuse, is a known side effect of agentic RL training, and it quietly inflates latency and cost in every deployed agent that suffers from it.

A new paper, "Learning When Not to Act: Mitigating Tool Abuse in Agentic Reinforcement Learning", tackles this problem head-on with a framework called EAPO (Efficient Agentic Policy Optimization). The core insight is deceptively simple: if you never show the model what it looks like to solve a problem without tools, it has no reference point for when tools are actually necessary. EAPO fixes that.
Why RL Makes Agents Tool-Happy
The dominant training algorithm for agentic models right now is GRPO (Group Relative Policy Optimization), popularized by DeepSeek-R1. GRPO is a sample-efficient algorithm that optimizes policies directly via groupwise outcome rewards, and its elimination of a value function approximation reduces training complexity while maintaining stability. It works well for reasoning, but when tools are added to the mix, a subtle failure mode emerges.
An AI agent can make twelve API calls to answer a question that needed two, with each unnecessary tool call burning tokens, adding latency, and pushing costs higher. The problem is structural: during RL training with tools available, the model learns that calling a tool is often a safe bet. It gets rewarded for correct answers, and tool calls frequently help it get there. Over time, the model over-indexes on tool use as a general strategy, even when internal reasoning would be faster, cheaper, and just as accurate.