Skip to main content
Back

Kyutai's AURA Generates Real-Time Talking-Head Video 23x Faster Than Rivals

Kyutai's AURA distills a bidirectional video diffusion teacher into a causal student that streams lip-synced talking heads live on one H100.

  • Kyutai released AURA, a streaming audio-driven talking-head diffusion model running real-time on one H100.
  • AURA 1.3B streams 576p at 20.7 fps; AURA 5B streams 720p at 9.7 fps using four denoising steps.
  • Chunk-wise causal attention with a rolling KV cache removes the multi-second audio look-ahead of bidirectional baselines.
  • Two-stage distillation combines ODE teacher forcing with Distribution Matching Distillation on the student's own rollouts.
  • Reported 23x and 17x speedups over the bidirectional TalkVerse teacher at matched or comparable quality on Hallo3 and CelebV-HQ.
  • Weights, code, and a 30K-video dataset with precomputed teacher trajectories are open-sourced on Hugging Face and GitHub.

Kyutai has released AURA, an open audio-to-video diffusion system that generates talking-head video in chunks using only the current audio chunk. The smaller model produces frames faster than Kyutai’s 16 fps playback target on one GPU, making it suitable for interactive avatars and visual interfaces for voice agents.

Existing systems such as Wan-S2V, Hallo3, and TalkVerse process multi-second audio windows and run dozens of denoising steps before producing output. That design works for offline generation but adds too much delay for a live conversation. AURA removes future-audio look-ahead and reduces sampling from 50 denoising steps to four.

AURA model configurations and reported throughput
Model Output Throughput Rate versus 16 fps playback
AURA 1.3B 576p 20.7 fps 1.29×
AURA 5B 720p 9.7 fps 0.61×

AURA 1.3B can sustain the stated playback rate, while AURA 5B exchanges throughput for 720p output. Both checkpoints were distilled from a bidirectional TalkVerse teacher that used 50 diffusion steps per chunk.

Comparison of AURA quality and generation speed with other talking-head video models
Kyutai compares AURA’s quality and generation speed with offline and streaming talking-head systems.

Four steps without future audio

Conventional audio-to-video diffusion models attend across an entire video chunk and its corresponding audio window during denoising. Generation therefore waits until the model has received the full window, including audio that occurs after the first frame it needs to render.

AURA replaces that bidirectional structure with causal, chunk-wise generation. Each chunk contains four latent frames, which are compressed video representations later decoded into pixels. A chunk can attend to its own frames, the preceding chunk, and the first chunk in the sequence. That first chunk acts as a stable attention anchor throughout the stream.