Skip to main content
Back

Xiaohongshu's Relax Open-Source Engine Trains Qwen3-Omni 2x Faster

Xiaohongshu's open-source RL engine hits 2x speedup on 30B omni-modal training with service-isolated roles and a single staleness knob spanning on-policy to fully async.

Pro

  • Xiaohongshu AI Infra open-sourced Relax, a service-oriented async RL engine for omni-modal post-training.
  • 1.20x speedup over veRL on Qwen3-4B, 1.76x async vs colocate, 2.00x on Qwen3-Omni-30B.
  • Single max_staleness knob spans on-policy, near-on-policy, and fully async without code changes.
  • R3 routing replay for MoE adds only 1.9% overhead versus 32% in veRL.
  • Stable convergence on Qwen3-Omni across image, text, audio, and 2000+ video steps.
  • Built on Ray Serve, Megatron-LM, and SGLang with Apache 2.0 license.

Reinforcement learning post-training has become the standard tool for coaxing reasoning and tool-use out of large language models, but the surrounding infrastructure is straining under two shifts: models now consume images, video, and audio alongside text, and rollouts have grown into multi-turn agentic trajectories with tool calls. Xiaohongshu's AI Infra team has released Relax, an open-source RL training engine that pairs Megatron-LM for training with SGLang for inference, fully decouples the two through a TransferQueue data bus, and supports end-to-end multimodal RL training across text, images, video, and audio.

The headline numbers: a 1.20x end-to-end speedup over veRL on Qwen3-4B on-policy training, 1.76x over colocate mode on Qwen3-4B, and 2.00x on Qwen3-Omni-30B, with every mode converging to the same reward.

Where the current stack breaks

Most existing RL frameworks were designed around text, and bolting multimodal support onto them tends to break in three connected ways. A single batch may mix image patches at varying resolutions, variable-length video, and audio waveforms with text tokens, forcing training systems to juggle heterogeneous data flows, operational robustness at scale, and the staleness/throughput tradeoff simultaneously. A high-resolution image can carry 50x the token count of its text prompt, and a 30-second video can take 10x longer to process than a text-only batch, which turns long-tail latency into a job killer.

At hundreds to thousands of GPUs running for days, hardware failures, NCCL timeouts, and inference-engine hangs become routine. A monolithic training loop restarts the whole job on any of these, wasting hours of computation. Synchronous execution leaves the trainer GPU idle waiting for the slowest rollout, while fully asynchronous execution risks training on stale policy data.