Skip to main content
Back

MBZUAI's KineVLA Teaches Robots to Follow Precise Motion Instructions, Not Just Goals

A new vision-language-action framework decouples task goals from motion details, letting robots follow fine-grained kinematic instructions like orientation, direction, and trajectory.

Pro

Paper, Full Access
  • KineVLA introduces a kinematics-rich VLA task where instructions densely encode direction, orientation, and trajectory details.
  • The framework uses a bi-level RVQ-VAE with separate goal and kinematics codebooks trained in two stages.
  • Bi-level reasoning tokens plus an InfoNCE mutual information loss align generated text with executed motion.
  • Three new datasets total 378K frames across LIBERO simulation and a Realman-75 real robot arm.
  • Randomizing reasoning tokens drops kinematics success by roughly 20 points while leaving goal success intact.
  • Inference runs at 96ms, roughly 2.5x faster than OpenVLA, with code and data promised.

Tell a robot to "place the wine bottle on the drawer," and most modern vision-language-action (VLA) models will happily oblige. Tell it to "place the wine bottle on the drawer with the label facing right," and things fall apart. A team at MBZUAI has published KineVLA, a framework that separates what a robot is supposed to do from how it should move while doing it, with meaningful gains on manipulation tasks where the difference matters.

Wine bottle label orientation comparison between vanilla VLA and KineVLA

The blind spot in current VLAs

Recent VLA models like OpenVLA and the pi_0 family have gotten good at goal-oriented manipulation, but they share a hidden assumption: instructions describing the same task goal can be executed with interchangeable motion trajectories. That assumption breaks the moment a human user cares about orientation, distance, or the shape of the path.

The authors trace this to what current models optimize for. Existing VLAs learn to determine what to do (grasp or place) but not how to do it (at which position, distance, or orientation). Even tokenization-based or diffusion-based decoders that smooth trajectories remain agnostic to fine-grained kinematic cues expressed in language. That gap is what KineVLA targets.

Two codebooks, two reasoning streams

The core idea is a bi-level action representation built on a Residual Vector-Quantized Variational Autoencoder (RVQ-VAE, a tokenizer that compresses continuous motions into discrete codes by repeatedly quantizing the leftover error). KineVLA learns two codebooks of equal size instead of one: a goal (coarse-grained) level that captures low-frequency structure and task goal, and a kinematics (fine-grained) level that captures high-frequency corrections and contact-rich transients.