UC Berkeley's VDN-H3 Generates 14-Second Videos Faster Than You Can Watch

A team from UC Berkeley, Impossible Inc., and UT Austin released weights and code for a hybrid attention rework of MiniMax H3 that generates video faster than you can watch it. Video DeltaNet (VDN-H3) is an open-source patch on top of the MiniMax H3 text-to-video backbone. On 8 B200 GPUs, it generates a 14.4-second 768p clip in 11.23 seconds across 8 denoising steps, a 75-to-90x speedup over the dense baseline, without the visible quality collapse that usually accompanies linear attention.

The release includes a Hugging Face checkpoint, an optimized inference stack, training code, and a technical blog. A community contributor has already shipped a native ComfyUI port, so the model is reachable outside the datacenter reference setup.

The attention bottleneck in video diffusion

Video diffusion transformers spend most of their compute inside self-attention, and costs scale quadratically with clip length. On MiniMax H3, softmax attention over long token sequences accounts for more than 85% of total runtime. Swapping in linear attention is the obvious remedy, but it carries steep quality tradeoffs: degraded subject consistency, broken scene layout, and poor long-range dependency capture.

VDN-H3 sidesteps the tradeoff by splitting video-video attention into two complementary branches. A sliding-window softmax branch handles local frame pairs, computing exact attention between nearby frames to preserve fine-grained detail and short-term temporal stability. Concretely, sets of five consecutive latent frames are grouped into chunks, each attending to itself plus the preceding and following chunks, following the original H3 architecture. A separate linear branch handles long-range context.