Alibaba's AutoDrive-R² Beats Rivals Using a Tenth of Their Training Data
A two-stage training recipe teaches vision-language-action driving models to reason step by step and self-correct with physics-grounded rewards.
PRO- AutoDrive-R² combines chain-of-thought SFT and GRPO reinforcement learning for VLA-based autonomous driving trajectory planning.
- New nuScenesR²-6K dataset uses a four-step Observation→Calculation→Logic→Reflection reasoning template with self-validation.
- Physics-grounded reward penalizes position error, steering angle, velocity, and temporal jerk during RL training.
- Achieves 0.19m average L2 error on nuScenes, beating EMMA+ with roughly 11% of its training data.
- Zero-shot Waymo evaluation shows 33.3% L2 error reduction versus EMMA+, and 89.1 PDMS on closed-loop NAVSIM.
- Full paper promises dataset, code, and checkpoints on acceptance; pure-RL training underperformed SFT+RL by 22.2%.
A team from AMAP (Alibaba Group) and collaborators has released AutoDrive-R², a vision-language-action (VLA) framework for autonomous driving that trains a model to predict trajectories, reason about them in structured steps, and then check its own work. On the nuScenes benchmark it cuts average trajectory error to 0.19 meters using roughly a tenth of the training data that competing systems consume.
Why end-to-end driving models keep tripping on physics
VLA models extend vision-language models with an action head, so a car can go from camera pixels and instructions directly to steering and waypoints. The authors identify a recurring failure: existing approaches that generate textual commands or waypoints via VLMs frequently produce physically infeasible outputs and exhibit model collapse. Workarounds like predicting meta-actions or latent action tokens help, but these designs violate the end-to-end optimization principle and significantly increase model complexity.
Thin reasoning is the second failure mode. Most driving VLMs use a single-shot prompt that maps image to trajectory, which breaks down when the scene involves kinematics (how tight a turn is physically possible, for example) or nuanced traffic logic. AutoDrive-R² attacks both problems with a two-stage training pipeline built on top of Qwen2.5-VL.
A chain-of-thought dataset that thinks like a driver
The first stage is supervised fine-tuning on a new dataset called nuScenesR²-6K, which contains 6,000 image-trajectory pairs enriched with structured reasoning traces. The dataset was built through a "generate-then-validate" pipeline in which Qwen2.5-VL-72B writes the reasoning chains and Qwen-VL-Max acts as a validator that discards chains with factual errors or logical inconsistencies.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
