Kyutai's CADRE Beats Rivals on Speed and Quality in Interactive World Models

Interactive world models generate video frames in response to keyboard input or an agent policy. Their autoencoders compress pixels into smaller latent representations that the simulator can process efficiently. Many inherit this component from text-to-video systems such as Wan or Cosmos, whose codecs were designed to render complete clips without waiting for live actions.

Kyutai’s CADRE project introduces seven causal autoencoders built for interactive generation. Across Kyutai’s single-GPU tests, CADRE provides most of the configurations for which no tested competitor is both faster and higher quality.

Why frame packing stalls control

Video autoencoders raise throughput by packing several consecutive frames into one latent token. An eight-frame token cannot be completed until the system has received all eight frames and their corresponding actions. Throughput can remain high while action-to-frame latency rises.

Offline clip generation can absorb this delay because the inputs are already available. Interactive simulators receive each action sequentially, so temporal packing can make a robot, driving agent, or game respond several frames late.

CADRE, short for Causal Diffusion REpresentation autoencoders, restricts each output to current and previous inputs. The model never relies on unseen future frames, and its compression profiles let developers control how much temporal waiting the codec introduces.

A frozen encoder, a one-step decoder

The paper describes an encoder based on MIRA, Kyutai’s earlier Rocket League world model. It uses frozen DINOv3 features with a small learned bottleneck. “Frozen” means the DINOv3 feature extractor remains unchanged while the bottleneck learns the compact video representation.

The decoder uses diffusion to reconstruct video from that representation. Regression decoders tend to average several plausible outcomes under heavy compression, producing blur. Diffusion can sample one plausible reconstruction, although its usual iterative process is expensive. CADRE self-distills that process into a single-step flow map for lower inference cost.

Seven variants, four test beds

The seven released models share the same architecture and differ in their spatial and temporal compression ratios. Kyutai trained the same downstream world-model recipe with each variant across four settings:

Compression leaves fingerprints

1. Diffusion preserves detail under heavy compression. Regression averages plausible reconstructions and softens the image. CADRE’s distilled diffusion decoder produces sharper samples while retaining single-step inference.
2. Temporal compression works best inside the autoencoder. Grouping frames within the world model degraded image quality and motion. Larger groups widened the gap, and eight-frame groups also weakened adherence to actions.
3. Camera motion changes the preferred profile. Fixed-camera manipulation can tolerate substantial temporal packing because consecutive frames remain similar. Moving-camera navigation benefits from lighter temporal packing and stronger spatial compression, which also reduces control latency.
4. Frame scores miss temporal failures. Coarser spatial tokens had little effect on individual manipulation frames while damaging motion and action-following. Fréchet Inception Distance, or FID, compares frame distributions and may overlook these rollout-level failures. Motion quality, action adherence, and temporal consistency need separate evaluation.

Camera motion reshapes the frontier

Kyutai compared CADRE with the autoencoders shipped with Wan 2.2, Cosmos, LTX-2, DC-AE, and MIRA. Every codec used the same downstream world-model recipe, helping isolate the effect of the latent representation and decoder.

CADRE variants occupy most of the measured latency-quality Pareto frontier, especially in moving-camera environments. A configuration sits on that frontier when no tested alternative improves one measured property without sacrificing another. Text-to-video codecs lose ground in this setting because clip-oriented temporal context may be unavailable at the current interactive step.

Where CADRE fits

The release includes training and inference code, seven pretrained models, and a browser demo that accepts keyboard controls. Developers can use the artifacts to:

• Train an interactive world model for robotics, driving, or embodied-agent research.
• Select heavier temporal compression for fixed-camera manipulation workloads.
• Select heavier spatial compression for navigation with a moving camera.
• Run policy evaluation or imitation-learning pipelines where each action needs a prompt visual response.
• Test the one-step diffusion decoder in latent-video systems affected by regression blur.
• Measure action-to-frame latency separately from frames per second.

Code, weights, and demo links are available on the project page. Kyutai also publishes artifacts through its GitHub organization and Hugging Face account.

Rollouts expose learned dynamics

Action-conditioned rollouts show unsupported sections of cloth falling, a towel folding, other cars moving independently, and users taking routes absent from the recorded driver trajectories. Those behaviors received no explicit labels; they emerged from video prediction conditioned on actions.

These examples provide qualitative evidence that the models capture useful dynamics. Controlled tests remain necessary to measure physical consistency, causal understanding, and performance outside the training distribution.

For neural-simulator teams, codec selection belongs in the core model design. Compression ratio determines waiting time and retained detail, causality controls dependence on future frames, and decoder architecture affects sharpness and compute cost. CADRE exposes those choices in an open family that can be evaluated through end-to-end latency, motion quality, and action fidelity on target hardware.