Ant Group's LingBot-VA Teaches Robots to Imagine Before Acting
Robbyant open-sources LingBot-VA, an autoregressive diffusion model that jointly predicts video futures and robot actions, beating π0.5 on both sim and real-world benchmarks.
PRO- What: Robbyant (Ant Group's embodied AI unit) open-sources LingBot-VA, a causal world model that jointly predicts video and robot actions.
- Architecture: Autoregressive diffusion model with a Mixture-of-Transformers backbone, closed-loop rollout, and asynchronous action execution.
- Benchmarks: Achieves 92.9% / 91.6% easy/hard success rate on RoboTwin 2.0, beating π0.5 (82.7% / 76.8%) and setting a new state-of-the-art.
- LIBERO: 98.5% average success rate, with 98.5% on the long-horizon subset vs π0.5's 92.4%.
- Access: Apache 2.0 license; weights on Hugging Face; requires ~24GB VRAM for inference.
- Paper: Accepted at RSS 2026; arXiv:2601.21998.
Most robot learning systems treat perception and action as separate problems: one model watches the world, another decides what to do. LingBot-VA, a new open-source release from Robbyant (the embodied AI arm of Ant Group), challenges that split. It is an autoregressive diffusion model that learns to predict future video frames and robot actions at the same time, in a single unified pass. The code, weights, and training pipeline are now publicly available under Apache 2.0, alongside the team's RSS 2026 paper.
The core idea: imagine before you act
This work argues that video world modeling establishes a fresh and independent foundation for robot learning, because video world models provide the ability to imagine the near future by understanding the causality between actions and visual dynamics. Rather than predicting actions directly from observations, LingBot-VA first imagines what the world will look like after an action, then uses that imagined future to inform what to do next.
The result is an autoregressive diffusion framework that learns frame prediction and policy execution simultaneously. The word autoregressive here means the model generates one chunk of video at a time, feeding each predicted chunk back as context for the next, building up a rolling mental simulation of the task as it executes.
Three design choices that make it work
The model features three carefully crafted designs: (1) a shared latent space integrating vision and action tokens, driven by a Mixture-of-Transformers (MoT) architecture, (2) a closed-loop rollout mechanism allowing for ongoing acquisition of environmental feedback with ground-truth observations, and (3) an asynchronous inference pipeline parallelizing action prediction and motor execution to support efficient control.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
