Zhejiang University's StaMo Teaches Robots Motion From Static Images, Boosting Success 30%
A new framework squeezes a robot's visual world into just two tokens, then discovers that subtracting them yields usable motion commands.
ProTakeaways
- StaMo compresses a robot camera image into just two 1024-dim tokens using a frozen DINOv2 encoder plus a Diffusion Transformer decoder.
- Motion emerges as the simple vector difference between two state tokens, no video or temporal model required.
- Boosts OpenVLA by 11.6% on LIBERO and lifts real-world success by 31% with negligible inference cost.
- Zero-shot transfer to ManiSkill more than doubles baseline success on Push Cube and Stack Cube tasks.
- Co-training with StaMo pseudo-actions outperforms LAPA and ATM by 10.4% on average across LIBERO suites.
- Code and models are open source at github.com/aim-uofa/StaMo under BSD-2.
A team from Zhejiang University and collaborators has released StaMo, a framework that challenges one of the load-bearing assumptions of modern robot learning: that you need video sequences and heavy temporal models to teach a policy what motion looks like. Compress a single image into a small enough latent state, they show, and motion falls out for free as the arithmetic difference between two of those states.
The method pairs a lightweight encoder with a pre-trained Diffusion Transformer (DiT) decoder to learn a highly compressed two-token state representation. The headline numbers: a 14.3% improvement on LIBERO, a 30% jump in real-world task success with minimal inference overhead, and a 10.4% gain over prior latent-action methods in policy co-training. The code is on GitHub under a 2-clause BSD license.

The representation trap in robot learning
Vision-language-action (VLA) models today face an awkward tradeoff. Feeding the policy raw feature maps from something like DINOv2 yields around 256 tokens per image, which is expressive but heavy and full of details irrelevant to picking up a cup. Compressing hard with a single CLS token or a low-dimensional trajectory throws away the semantic and geometric structure the policy needs. Motion representations like optical flow or end-effector deltas are compact but strip out context; visual features are rich but bloated.
A parallel line of work tries to extract latent actions from unlabeled video, treating action learning as a form of self-supervised temporal modeling. Systems like LAPA and UniVLA rely on this approach, but the authors argue it carries real costs. Extracting frames from continuous videos introduces sensitivity to sampling intervals, temporal biases, and poorly interpretable, blurry action representations. Most approaches also lean on carefully designed model architectures. The core question StaMo asks: what if a good enough state representation makes the temporal machinery unnecessary?