NVIDIA's LongLive-2.0 Generates 64-Second 720p Video Using 6x Less Memory
NVIDIA's LongLive-2.0 bakes NVFP4 quantization into every stage of long video generation, delivering 2.1x faster training and 45.7 FPS at 720p on Blackwell GPUs.
Takeaways
- First end-to-end NVFP4 system: LongLive-2.0 is the first pipeline to apply 4-bit FP4 quantization across training, distillation, and inference for long video generation.
- Major speed gains: 2.15x faster training and 1.84x faster inference vs BF16 baseline; 45.7 FPS at 720p on GB200.
- Dramatic memory reduction: 64-second 720p video generated using only 19.4 GB peak memory, vs 112.9 GB for BF16 baseline.
- Fully open-source: Weights, code, and datasets released under NVIDIA Open Model License in base, 4-step, and 2-step variants.
- Hardware caveat: Full NVFP4 acceleration requires Blackwell GPUs (GB200); older hardware falls back to multi-GPU sequence parallelism.
- Top VBench-Long ranking: LongLive-2.0 ranks first by average rank on the VBench-Long 60-second benchmark while maintaining BF16-comparable quality.
Long video generation has a dirty secret: the algorithmic breakthroughs get the headlines, but the real bottleneck is infrastructure. Memory explodes, training stalls, and inference pipelines buckle under the weight of minute-long sequences. NVIDIA Research's LongLive-2.0 takes direct aim at this, treating the entire stack -- training, distillation, and inference -- as a single system built around NVFP4, NVIDIA's 4-bit floating-point format.
The Problem Nobody Talks About
Most video generation systems are trained in high precision (BF16, which stores each number in 16 bits) and then quantized down to lower precision at deployment time. Quantization compresses model weights by reducing the number of bits used to represent each number: BF16 stores each number in 16 bits, while NVFP4 compresses them to 4 bits, so the model uses less memory and runs faster. The catch is that this post-training quantization creates a gap -- the model was never optimized for the precision it actually runs at, and quality degrades as a result.
LongLive-2.0 presents an NVFP4-based parallel infrastructure throughout the full training and inference workflow of long video generation, addressing speed and memory bottlenecks. The key insight is that if you design for NVFP4 from the very first training step, you close that gap entirely. In theory this should reduce generation quality, but the authors show that in practice results are nearly identical to BF16.
There is also a second, less obvious problem. Instead of focusing purely on algorithmic novelty, the paper presents a systems-level approach to tackle the speed and memory bottlenecks in long video generation. For a 64-second video, the memory and compute requirements don't grow linearly -- they compound. Plain BF16 is efficient only at shorter video lengths, taking 75.3s and 202.7s at 16s and 32s, but running out of memory entirely at 64s.
How It's Built
LongLive-2.0 treats algorithm and infrastructure as one system. On the training side, Balanced SP and NVFP4 make long-video AR fine-tuning practical. On the inference side, W4A4 execution, NVFP4 KV cache, parallel dequantization, and asynchronous VAE decoding improve end-to-end throughput.
The training side introduces a concept called Balanced Sequence Parallelism (Balanced SP) -- a way to split long video sequences across multiple GPUs during autoregressive (AR) training. AR training here means the model generates video chunk by chunk, each new chunk conditioned on the previous ones (like a language model generating tokens). The challenge is that standard sequence parallelism doesn't map cleanly onto this structure. Each SP rank owns clean and noisy latents from the same temporal chunk. This matches the teacher-forcing attention mask, balances loss-bearing tokens across ranks, and lets VAE encoding be sharded by chunk instead of replicated on every GPU.
The training pipeline is also notably simpler than prior work. Unlike existing Self-Forcing series methods that rely on ODE initialization and subsequent distribution matching distillation (DMD), LongLive-2.0 directly tunes a diffusion model into a long, multi-shot, interactive autoregressive diffusion model. It can be further converted to real-time generation (4 to 2 denoising steps) with standalone LoRA weights. LoRA (Low-Rank Adaptation) here means only a small set of extra weights are trained on top of the frozen quantized backbone -- keeping the distillation step cheap and stable.

On the inference side, the system stacks several optimizations together: W4A4 NVFP4 inference, KV cache quantized into NVFP4 for memory savings, and end-to-end throughput boosted with asynchronous streaming VAE decoding. W4A4 means both the weights and the activations (the intermediate values computed during a forward pass) are stored in 4-bit precision. The KV cache -- the stored attention context that grows with video length -- is also compressed to NVFP4. And VAE decoding (converting the model's latent representation back into actual pixels) is offloaded asynchronously to a second GPU, overlapping with the next denoising step.
For multi-shot consistency across scene changes, the paper introduces a Multi-Shot Attention Sink. A fixed global sink preserves identity across the full video, while a shot-level sink is rebound at every scene change to maintain local temporal coherence. This supports minute-scale streaming without recomputing earlier history.
The Numbers
The headline results are significant. A 5 billion parameter model generates video at 1280x720 resolution at 45.7 frames per second -- the first time NVFP4 has been applied across the full training and inference pipeline for long video generation. For context, 45.7 FPS at 720p is faster than real-time playback.

The end-to-end latency improvements are equally striking:
- A 16-second video: BF16 baseline takes 26.6s; LongLive-2.0 2-step takes 11.2s
- A 32-second video: BF16 takes 53.2s; LongLive-2.0 takes 19.2s
- A 64-second video: BF16 takes 112.9s; LongLive-2.0 takes 36.3s
The 2-step denoising model generates a 64-second 720p video in 36.3 seconds while using 19.4 GB of GPU memory. By comparison, the BF16 version with 4 steps required 112.9 GB. That is a nearly 6x reduction in peak memory for the same output length.
On the VBench-Long benchmark (60-second video generation), LongLive-2.0 ranks first by average rank among all compared methods. Quality is preserved even at 4-bit precision, validating the core claim that training-aware quantization avoids the degradation seen in post-training approaches.
The Hardware Catch
There is a real constraint worth flagging. The acceleration gain from low-bit quantization is hardware-dependent. NVFP4 inference delivers acceleration only on Blackwell GPUs (e.g., GB200), which are equipped with the latest-generation Tensor Cores and optimized kernels. On non-Blackwell GPU architectures, the system deploys SP inference to match the speed on Blackwell GPUs, while the quantized KV cache can lower inter-GPU communication overhead. So the 45.7 FPS headline number requires a GB200 -- older hardware will need multi-GPU setups to get close.
What It Unlocks
The practical use-cases this enables are meaningful:
- Multi-shot narrative video: The prompt-switching at chunk boundaries enables cinematic sequences with consistent characters across scene changes
- Interactive video generation: Low latency makes real-time or near-real-time generation viable for interactive applications
- Research at reduced cost: Training is 2.15x faster and inference is 1.84x faster compared to the BF16 baseline, making long-video research accessible without massive GPU clusters
- Streaming video pipelines: Asynchronous VAE decoding means video can be streamed out while the next chunk is still being generated
Open and Available Now
LongLive-2.0 is available as an open model in three forms: the Base Model, the NVFP4 Quantized 4-Step Generative Model, and the NVFP4 Quantized 2-Step Generative Model. The license is the NVIDIA Open Model License. Model weights, code, and datasets are available on GitHub and Hugging Face. The paper is on arXiv with full technical details.
The broader implication for the field is a shift in how video generation systems should be designed. The assumption that you train in high precision and quantize later is increasingly hard to defend when the quantization-aware approach is faster, cheaper, and matches quality. LongLive-2.0 makes the case that the training stack and the deployment stack need to be designed together from the start -- and that for long video specifically, there is no other practical path forward.