Skip to main content
Back

ByteDance's FTRL Trains Tool-Calling AI to Beat Claude Without Live APIs

Researchers from ByteDance Seed and Fudan propose FTRL, a pipeline that builds local tool-use training environments with verifiable rewards.

  • ByteDance Seed and Fudan researchers introduce FTRL, an automated pipeline for building local tool-use RL environments.
  • Five-stage construction covers scenario decomposition, document generation, function merging, complexity scaling, and local Python deployment.
  • F1-inspired reward balances tool-call precision and task completion using only environment feedback, no LLM judge.
  • Trained 7B–14B Qwen models gain 10+ points average across ToolHop, τ-bench, RoTBench, and in-domain tests.
  • Parameter analysis shows most learning concentrates in lower-layer MLP weights, suggesting improved context representation.
  • Reasoning-mode Qwen3 models underperform non-reasoning ones on simple tool tasks, exposing a misalignment gap.

Getting large language models to reliably call external tools has become one of the messier problems in agent research. You need environments to train them in, reward signals that actually reflect good behavior, and enough diversity to avoid overfitting. Most existing pipelines lean on live web APIs, which break, rate-limit, and behave inconsistently. A new paper from ByteDance Seed and Fudan University, accepted to ACL 2026 Findings, takes a different route by building the environments from scratch, locally, in a fully automated way.

The framework is called FTRL (Feedback-driven Tool-use Reinforcement Learning), and the authors have open-sourced both code and data. Their core argument: RL progress on tool use has been limited by the difficulty of constructing stable training environments and designing verifiable reward mechanisms. Their answer is an automated pipeline that handles scenario decomposition, document generation, function integration, complexity scaling, and localized deployment, producing training environments that give detailed, measurable feedback without relying on external services.

Why external toolsets keep breaking RL for agents

Tool-use training has typically depended on scraping large collections of real-world APIs and pointing models at them. That setup has two chronic failure modes. First, the tools themselves are unreliable: rate limits, outages, and version drift make reward signals noisy. Second, judging whether a multi-step trajectory was actually correct usually requires an LLM-as-judge, which introduces bias and instability during training.

FTRL sidesteps both problems by generating tools as local Python functions, each tied to a known sub-question with a known answer. Every step of a rollout can be checked deterministically against ground truth, and no network calls are required during training.