Skip to main content
Back

ReSkill Lets AI Agents Build and Reuse Strategies, Hitting 95.3% on Unseen Tasks

Researchers introduce ReSkill, an open-source RL framework that keeps skill libraries in sync with a learning policy, beating baselines by up to 12.7% on unseen tasks.

  • Researchers publish ReSkill, an RL framework that keeps a skill library synchronized with a learning policy during GRPO training.
  • Prior skill-augmented RL methods update skills separately from policy optimization, causing skill-policy mismatches that hurt generalization.
  • ReSkill embeds three mechanisms into existing GRPO rollouts: an assertion-driven skill creator, within-group A/B skill testing, and Thompson Sampling for version selection.
  • On ALFWorld (8B), ReSkill hits 95.3% on unseen tasks, a 12.7-point gap over the SkillRL baseline.
  • The co-evolved policy can adapt to entirely new domains at test time via skill injection alone, without retraining.
  • Code is open-sourced under Apache 2.0 as a veRL extension with configurable skill and policy components.

When you train an LLM agent with reinforcement learning, it gets better at the tasks it sees during training. But ask it to generalize to something new, and it often struggles to apply what it has learned. The agent has no structured way to package and reuse the strategies it discovers. A new paper from Zelin He et al. introduces ReSkill, a framework that fixes this by building a living library of reusable skills that evolves in lockstep with the policy itself.

The skill-policy mismatch problem

Agentic RL enables LLM agents to improve continuously from environment rewards, yet the resulting policies do not systematically accumulate reusable strategies that generalize across tasks. The natural fix is to maintain a library of modular skills -- named, reusable strategies the agent can call on when it recognizes a familiar situation. But there is a catch.

Existing skill-augmented RL methods decouple skill creation from policy optimization, risking adopting skills that conflict with the evolving policy. In other words, skills are typically written or updated on a separate schedule from training. By the time a skill gets promoted into the library, the policy may have already moved on to a different behavioral regime where that skill is actively harmful. This is the mismatch ReSkill is designed to eliminate.

RL-in-the-loop skill creation

Inspired by Anthropic's Skill Creator, ReSkill is an RL-in-the-loop skill creation framework that reconciles skill evolution with policy learning. The key insight is to use the same rollout batches that GRPO (Group Relative Policy Optimization -- an RL algorithm that compares groups of sampled trajectories to compute a relative advantage signal, popularized by DeepSeek-R1) already generates for gradient updates, and extract skill-related signals from them for free.