OpenPipe's Open-Source ART Trains AI Agents Without Writing Reward Functions

OpenPipe has open-sourced Agent Reinforcement Trainer (ART), a Python framework for fine-tuning LLM-based agents with reinforcement learning without hand-writing reward functions or managing a training cluster. ART wraps techniques like GRPO (Group Relative Policy Optimization) to improve model performance while keeping training costs low. The repo is Apache-2.0 and has crossed 9k stars.

Why bother with another RL trainer

Training agents with RL is normally painful for two reasons: you have to write a reward function that captures what a good trajectory looks like, and you have to babysit vLLM plus a training loop on GPUs while your agent code runs somewhere else. ART removes both burdens, with current support for GRPO and planned support for PPO and other techniques, aimed at training efficiency and multi-turn agentic workflows.

The design choice that makes this practical is a clean split between where your agent runs and where training happens. The GRPO loop is divided into a frontend of mostly user-defined code for rollouts and reward computation, and a backend responsible for LLM inference and training. These can live on separate machines, so the frontend runs in your local environment with all its tooling while the backend runs on a cloud GPU. You keep writing normal Python that calls an OpenAI-compatible endpoint, and ART handles LoRA training in the background.