UCLA's FetchMan Trains Humanoid Robots to Fetch Objects With 73% Real-World Success
A UCLA and Allen AI team trained a humanoid to walk up and grab objects using only simulated data, hitting 73.3% real-world success on a Unitree G1.
Takeaways
- FetchMan trains a vision-based humanoid policy entirely in simulation and deploys zero-shot to a real Unitree G1.
- Behavior cloning on 350,000 synthetic episodes saturated at 67% success no matter how much data was added.
- Flow-GRPO reinforcement learning on a single sparse reward lifted simulation success to 83% and real-world to 73.3%.
- Two design choices, frozen DINOv3 encoder and delta-action targets, are non-negotiable, swapping either drops real-world success to 0%.
- All Flow-GRPO gains land on locomotion; manipulation alone barely moves from 72.7% to 77.2%.
- Team releases code, data and FetchMan-Bench, a humanoid loco-manipulation simulation benchmark built on MolmoSpaces.
Getting a humanoid robot to walk across a room, spot the object you asked for, and pick it up sounds simple until you count the demonstrations it would take to teach one by imitation. A new paper from UCLA and Allen AI called FetchMan skips real-world teleoperation entirely. The team trained a vision-based policy purely in simulation across more than 150,000 scenes, then dropped it onto a real Unitree G1 with no fine-tuning and watched it reach and grasp objects across unseen rooms at 73.3% success.
Behavior cloning alone hit a wall no amount of extra data could push past. The fix was a reinforcement learning stage using a variant called Flow-GRPO on a single sparse reward, and that combination is what closed the sim-to-real gap.
Why cloning synthetic demos hits a ceiling
Locomotion policies have transferred from simulation to the real world for years. Manipulation has not, largely because collecting enough tabletop demonstrations is expensive, and collecting them for a robot that also has to balance on two legs is worse. The obvious workaround is to script a demonstrator in simulation and generate as many episodes as you want. The FetchMan team did exactly that, producing 350,000 episodes in two days across Allen AI's MolmoSpaces scene dataset.
Then cloning those demos saturates. On the single-object fetch task, behavior cloning peaked at 67% success and refused to improve whether they trained on 5k, 50k, or 150k demonstrations. On the multi-object version it topped out around 40%. Pouring more synthetic data into a supervised loss does not break through, because the policy inherits every quirk of the scripted demonstrator and averages over the noise instead of learning to recover from it.