Skip to main content
Back

EAPO Fixes AI Agents That Compulsively Call Tools, Boosting Accuracy by 10%

EAPO teaches RL-trained agents to skip unnecessary tool calls, boosting accuracy by up to 10% while cutting tool use by up to 25%

  • Tool abuse problem: RL-trained agents overuse tools like web search and code interpreters even for simple queries solvable by internal reasoning.
  • EAPO framework: Introduces tool-free trajectories, difficulty-aware reward penalties, and confidence-aware token reweighting to fix this during training.
  • Key results: Up to +10.45% accuracy improvement over GRPO while reducing tool calls by up to 24.59% across three model families.
  • Models tested: Qwen2.5-3B, Qwen2.5-7B, and Llama3.1-8B evaluated on nine math and knowledge-intensive benchmarks.
  • Practical impact: Better accuracy AND fewer tool calls simultaneously, disproving the assumption that restricting tools hurts performance.
  • Availability: No code or weights released yet; the paper provides sufficient detail for reimplementation on top of existing GRPO pipelines.

When you give a language model access to tools like a code interpreter or a web search engine, something strange happens during reinforcement learning training: the model starts calling those tools compulsively, even for questions it could easily answer from memory. Ask it what 1+1 equals and it might spin up a Python interpreter. Ask it which direction the sun rises and it fires off a web search. This behavior, called tool abuse, is a known side effect of agentic RL training, and it quietly inflates latency and cost in every deployed agent that suffers from it.

GRPO model using Python for 1+1 and web search for sunrise direction, illustrating tool abuse

A new paper, "Learning When Not to Act: Mitigating Tool Abuse in Agentic Reinforcement Learning", tackles this problem head-on with a framework called EAPO (Efficient Agentic Policy Optimization). The core insight is deceptively simple: if you never show the model what it looks like to solve a problem without tools, it has no reference point for when tools are actually necessary. EAPO fixes that.

Why RL Makes Agents Tool-Happy

The dominant training algorithm for agentic models right now is GRPO (Group Relative Policy Optimization), popularized by DeepSeek-R1. GRPO is a sample-efficient algorithm that optimizes policies directly via groupwise outcome rewards, and its elimination of a value function approximation reduces training complexity while maintaining stability. It works well for reasoning, but when tools are added to the mix, a subtle failure mode emerges.

An AI agent can make twelve API calls to answer a question that needed two, with each unnecessary tool call burning tokens, adding latency, and pushing costs higher. The problem is structural: during RL training with tools available, the model learns that calling a tool is often a safe bet. It gets rewarded for correct answers, and tool calls frequently help it get there. Over time, the model over-indexes on tool use as a general strategy, even when internal reasoning would be faster, cheaper, and just as accurate.