NVIDIA Backs DFlash to Boost LLM Inference 15x on Blackwell
NVIDIA's inference stack now supports DFlash, a block-diffusion speculative decoding method that delivers up to 15x throughput gains on Blackwell GPUs with zero code changes.
Takeaways
- NVIDIA has integrated DFlash speculative decoding into SGLang, vLLM, and TensorRT-LLM, delivering up to 15x throughput on Blackwell GPUs.
- DFlash replaces autoregressive draft models with a block-diffusion drafter that proposes an entire token block in one forward pass.
- KV injection from the target model conditions the drafter at every layer, achieving higher acceptance rates than EAGLE-3 with lower drafting cost.
- 20 model checkpoints are available on Hugging Face covering Llama, Gemma, Qwen, Kimi K2.6, and gpt-oss families.
- Adoption is drop-in: change a config flag and swap the draft checkpoint, no application code changes required.
- DFlash reframes diffusion LLMs as lightweight speculative drafters rather than standalone generators, a potentially broad shift in how diffusion models are used.
NVIDIA has published a full technical endorsement of DFlash, a speculative decoding technique that can push LLM inference throughput up to 15x higher on Blackwell hardware. More importantly, NVIDIA's inference engineering teams have done the integration work to bring DFlash into SGLang, vLLM, and TensorRT-LLM, meaning you can drop it into your existing serving stack without touching your application code.
The problem with how speculative decoding works today
Speculative decoding is a well-established trick for accelerating LLM inference. A small, fast draft model proposes several tokens ahead, and the large target model verifies them all in one parallel pass. If the draft was right, you get multiple tokens for the cost of one verification step.
The catch is that most draft models, including the popular EAGLE series, still generate their proposals autoregressively -- one token at a time. That means the drafting step itself is sequential, which caps how much parallelism you can actually extract. Methods like EAGLE and the native multi-token prediction modules in Gemma 4 and DeepSeek still rely on sequential autoregression in the draft model, generating tokens one-by-one -- a poor fit for modern hardware and a hard limit on achievable speedup.
What DFlash does differently
DFlash is an open source lightweight block diffusion model designed for speculative decoding. Its drafter generates an entire block of candidate tokens in a single forward pass, turning sequential drafting into block-parallel GPU work while preserving the target model's output quality through verification.
The architecture has three key components working together:
- Block-diffusion drafting: Instead of predicting the next token, the draft model predicts a whole block of masked future tokens simultaneously in one forward pass.
- Target hidden-state conditioning: DFlash utilizes the target model's hidden features as context, conditioning the draft model to predict future blocks of tokens in parallel. Large autoregressive LLMs implicitly encode information about multiple future tokens in their internal representations.
- KV injection: Unlike previous work, DFlash injects target model features directly into the draft model's KV cache. This scales better with increased draft depth, and allows the draft model to skip modeling the full context from scratch and focus purely on predicting the next block of tokens.
The result is a drafter that is both faster to run and more accurate. By confining diffusion to the drafting stage and conditioning on target-model features, DFlash achieves both high acceptance rates and low drafting latency, pushing speculative decoding to over 6x lossless speedup.
How it's trained
DFlash draft models are trained to align block-level diffusion predictions with the outputs of a frozen autoregressive target model. The design is deliberately lean: the draft model reuses the embedding and LM head from the target model, with only the few intermediate layers trained, keeping the parameter count minimal. This means a DFlash drafter for a given model is a small adapter, not a full second model to host.
What the numbers look like on Blackwell
DFlash increases inference performance for gpt-oss-120b on NVIDIA Blackwell by up to 15x at the same interactivity level. That peak figure comes from the high-interactivity range, where users expect 500-600 tokens per second. But the gains are consistent across the board:
| Model | Framework | Task | Speedup vs. Autoregressive |
|---|---|---|---|
| gpt-oss-120b | TensorRT-LLM | Coding (avg) | 2.6x |
| Llama 3.1 8B | TensorRT-LLM | Multilingual | 2.4x |
| Gemma 4 31B | vLLM | Math500 | 5.8x |
| Qwen3 8-B | SGLang | Math500 | 5.1x |
By confining diffusion models to the drafting stage, DFlash exploits their inherent parallelism while avoiding the quality degradation that has limited their standalone use. The target model still does verification, so output quality is mathematically guaranteed to match the original model.
Compared directly to EAGLE-3, the current state-of-the-art speculative decoding method, DFlash consistently wins. DFlash achieves over 6x lossless acceleration across a range of models and tasks, delivering up to 2.5x higher speedup than EAGLE-3.
NVIDIA's role: from research to production stack
Researchers at UC San Diego released the DFlash paper in February 2026 as part of ongoing work on faster, more efficient LLM inference on NVIDIA Blackwell. NVIDIA and the open source inference community helped ensure strong framework support across both SGLang and vLLM, giving developers a clear path to introduce DFlash into inference deployments on their serving stack of choice.
This is the announcement: NVIDIA has validated DFlash's performance on its hardware and done the engineering work to make it production-ready across its entire inference ecosystem. The research team has released 20 DFlash model checkpoints on Hugging Face with Blackwell and Hopper recipes, covering model families including Qwen, Kimi K2.6, Llama, Gemma, and gpt-oss.
Real-world adoption is already happening. Xiaomi's MiMo v2.5-Pro-UltraSpeed uses DFlash to achieve over 1k output tokens per second. The LMSYS team also jointly released a DFlash draft model for Qwen 3.5 397B-A17B, which outperforms native MTP speculation across all tested concurrency levels.
Drop-in adoption across three frameworks
The integration story is the most immediately practical part of this announcement. Each framework has a slightly different path, but all require minimal changes:
- vLLM: Swap your EAGLE-3 checkpoint for a DFlash checkpoint in the config. The integration runs through the open source Speculators library, which connects the DFlash drafter to the target model's hidden states inside the vLLM inference path.
- SGLang: Update the speculative decoding algorithm flag to
DFLASHand point it at the matching draft model checkpoint. SGLang's new Spec V2 engine also adds overlap scheduling that eliminates host-device synchronization overhead, adding another 33% on top of DFlash's gains. - TensorRT-LLM: Fully supported for Blackwell Ultra systems, with benchmarks run on eight-GPU DGX B300 configurations.
For SGLang, the launch command looks like this:
python -m sglang.launch_server \
--model-path Qwen/Qwen3.5-397B-A17B \
--speculative-algorithm DFLASH \
--speculative-draft-model-path modal-labs/Qwen3.5-397B-A17B-DFlash \
--speculative-dflash-block-size 8 \
--tp-size 8Why this matters beyond the benchmark
Beyond empirical improvements, DFlash suggests a new development paradigm for diffusion LLMs. Rather than competing with autoregressive models in end-to-end generation quality, diffusion models can serve as lightweight, specialized drafters optimized for fast and accurate block prediction.
This reframing is significant. The field has spent years trying to get diffusion-based language models to match autoregressive quality at generation time, with limited success. DFlash sidesteps that entirely by giving diffusion models a job they are architecturally well-suited for: fast, parallel, approximate prediction that gets verified anyway. Instead of training massive diffusion LLMs to match autoregressive quality, we can train lightweight diffusion adapters optimized for fast, accurate block prediction, with speculative verification guaranteeing output quality.
DFlash is free, open source, and available now. The model checkpoints are on Hugging Face and the framework integrations are live in current releases of SGLang, vLLM, and TensorRT-LLM.