NVIDIA's GRAIL Trains Humanoid Robots With 20,000 Fully Virtual Demos
NVIDIA's GRAIL generates 20,000+ robot training sequences entirely in simulation using video AI, then deploys them on a real humanoid with 84–90% success rates
Takeaways
- What it is: GRAIL is a fully digital pipeline from NVIDIA/UCLA that generates humanoid robot training data using 3D assets and AI video generation, with no physical robot needed until deployment. [Paper]
- Scale: Produces 20,000+ motion sequences covering pick-up, whole-body manipulation, sitting, and terrain traversal across 1,000 object assets and 1,000 terrain configs.
- Key innovation: Builds the 3D scene before generating video, so camera, scale, object geometry, and robot proportions are all known — eliminating the depth and scale ambiguity that plagues video-based reconstruction.
- Real-world results: Policies trained only on GRAIL-generated data achieve 84% success on diverse object pick-up and 90% on stair-climbing on a real Unitree G1 humanoid.
- Benchmarks: 88.9% physics-based tracking success rate vs. 24% for the next best baseline; 81.4% manipulation success vs. ~49% for prior methods; preferred by 74.7% of users in a perceptual study.
- Availability: Dataset on Hugging Face, code on GitHub under a non-commercial NVIDIA license; each motion includes the synthetic video, 4D reconstruction, and retargeted robot trajectory.
Training a humanoid robot to pick up objects or climb stairs normally means building physical setups, wiring up actors with motion-capture suits, and teleoperating the robot through every scenario you want it to learn. That process is expensive, slow, and nearly impossible to scale across the thousands of objects and terrain types a real-world robot will encounter. Humanoid robotics is hitting a data wall: teleoperation and motion capture took the field far, but they don't scale to every object, terrain, and behavior.
A team from NVIDIA and UCLA has published GRAIL, a digital generation pipeline that remains fully virtual until deployment, composing 3D assets, simulator-ready scenes, and priors from video foundation models to synthesize interactions without rebuilding physical environments or teleoperating the robot. The result is a dataset of over 20,000 motion sequences covering pick-up, whole-body manipulation, sitting, and terrain traversal, all generated without a single physical robot demonstration.

The core insight: build the 3D world first, then generate the video
Most prior work that uses video to train robots tries to reconstruct 3D motion from unconstrained, in-the-wild footage. That approach runs into a fundamental problem: a single 2D video doesn't tell you how far away objects are, what the exact camera angle is, or how big the person in the frame actually is. Every one of those unknowns introduces ambiguity that makes the recovered motion less useful for a robot.
GRAIL starts from fully specified 3D configurations in which object geometry, camera parameters, metric scale, environment depth, and a robot-proportioned character are known before video generation and reused during reconstruction. By locking down all of those variables upfront, the reconstruction problem becomes well-constrained. Instead of starting with an unconstrained video and trying to infer the 3D world, GRAIL builds the 3D world first and then generates the video.
How the pipeline works, step by step
The pipeline has three major stages. The first stage assembles a photorealistic 3D scene and generates a reference video of a human interacting with an object. The second stage reconstructs the precise 4D motion (positions over time in 3D space) of both the human and the object from that video. The third stage converts those human motions into robot-executable actions and trains policies that generalize across many objects and terrains.