Kyutai's AURA Generates Real-Time Talking-Head Video 23x Faster Than Rivals
Kyutai's AURA distills a bidirectional video diffusion teacher into a causal student that streams lip-synced talking heads live on one H100.
Takeaways
- Kyutai released AURA, a streaming audio-driven talking-head diffusion model running real-time on one H100.
- AURA 1.3B streams 576p at 20.7 fps; AURA 5B streams 720p at 9.7 fps using four denoising steps.
- Chunk-wise causal attention with a rolling KV cache removes the multi-second audio look-ahead of bidirectional baselines.
- Two-stage distillation combines ODE teacher forcing with Distribution Matching Distillation on the student's own rollouts.
- Reported 23x and 17x speedups over the bidirectional TalkVerse teacher at matched or comparable quality on Hallo3 and CelebV-HQ.
- Weights, code, and a 30K-video dataset with precomputed teacher trajectories are open-sourced on Hugging Face and GitHub.
Kyutai has released AURA, an open audio-to-video diffusion system that generates talking-head video in chunks using only the current audio chunk. The smaller model produces frames faster than Kyutai’s 16 fps playback target on one GPU, making it suitable for interactive avatars and visual interfaces for voice agents.
Existing systems such as Wan-S2V, Hallo3, and TalkVerse process multi-second audio windows and run dozens of denoising steps before producing output. That design works for offline generation but adds too much delay for a live conversation. AURA removes future-audio look-ahead and reduces sampling from 50 denoising steps to four.
| Model | Output | Throughput | Rate versus 16 fps playback |
|---|---|---|---|
| AURA 1.3B | 576p | 20.7 fps | 1.29× |
| AURA 5B | 720p | 9.7 fps | 0.61× |
AURA 1.3B can sustain the stated playback rate, while AURA 5B exchanges throughput for 720p output. Both checkpoints were distilled from a bidirectional TalkVerse teacher that used 50 diffusion steps per chunk.
Four steps without future audio
Conventional audio-to-video diffusion models attend across an entire video chunk and its corresponding audio window during denoising. Generation therefore waits until the model has received the full window, including audio that occurs after the first frame it needs to render.
AURA replaces that bidirectional structure with causal, chunk-wise generation. Each chunk contains four latent frames, which are compressed video representations later decoded into pixels. A chunk can attend to its own frames, the preceding chunk, and the first chunk in the sequence. That first chunk acts as a stable attention anchor throughout the stream.