Skip to main content
Back

Alibaba's AutoDrive-R² Beats Rivals Using a Tenth of Their Training Data

A two-stage training recipe teaches vision-language-action driving models to reason step by step and self-correct with physics-grounded rewards.

  • AutoDrive-R² combines chain-of-thought SFT and GRPO reinforcement learning for VLA-based autonomous driving trajectory planning.
  • New nuScenesR²-6K dataset uses a four-step Observation→Calculation→Logic→Reflection reasoning template with self-validation.
  • Physics-grounded reward penalizes position error, steering angle, velocity, and temporal jerk during RL training.
  • Achieves 0.19m average L2 error on nuScenes, beating EMMA+ with roughly 11% of its training data.
  • Zero-shot Waymo evaluation shows 33.3% L2 error reduction versus EMMA+, and 89.1 PDMS on closed-loop NAVSIM.
  • Full paper promises dataset, code, and checkpoints on acceptance; pure-RL training underperformed SFT+RL by 22.2%.

A team from AMAP (Alibaba Group) and collaborators has released AutoDrive-R², a vision-language-action (VLA) framework for autonomous driving that trains a model to predict trajectories, reason about them in structured steps, and then check its own work. On the nuScenes benchmark it cuts average trajectory error to 0.19 meters using roughly a tenth of the training data that competing systems consume.

Why end-to-end driving models keep tripping on physics

VLA models extend vision-language models with an action head, so a car can go from camera pixels and instructions directly to steering and waypoints. The authors identify a recurring failure: existing approaches that generate textual commands or waypoints via VLMs frequently produce physically infeasible outputs and exhibit model collapse. Workarounds like predicting meta-actions or latent action tokens help, but these designs violate the end-to-end optimization principle and significantly increase model complexity.

Thin reasoning is the second failure mode. Most driving VLMs use a single-shot prompt that maps image to trajectory, which breaks down when the scene involves kinematics (how tight a turn is physically possible, for example) or nuanced traffic logic. AutoDrive-R² attacks both problems with a two-stage training pipeline built on top of Qwen2.5-VL.

Two-stage AutoDrive-R2 training pipeline with CoT SFT and physics-grounded reinforcement learning

A chain-of-thought dataset that thinks like a driver

The first stage is supervised fine-tuning on a new dataset called nuScenesR²-6K, which contains 6,000 image-trajectory pairs enriched with structured reasoning traces. The dataset was built through a "generate-then-validate" pipeline in which Qwen2.5-VL-72B writes the reasoning chains and Qwen-VL-Max acts as a validator that discards chains with factual errors or logical inconsistencies.