NVIDIA Shrinks DeepSeek V4 Flash to Run on a Single Blackwell Node

NVIDIA has published an official NVFP4 quantization of DeepSeek V4 Flash 0731, the July refresh of DeepSeek's efficiency-tier Mixture-of-Experts model. The checkpoint is built with NVIDIA's Model Optimizer toolkit and targets Blackwell-class GPUs, where FP4 tensor cores execute 4-bit math natively rather than upcasting to FP8 or BF16.

DeepSeek V4 Flash 0731 has 284B total parameters with 13B activated per token and a 1M-token context window. Getting that into VRAM on a reasonable node is the whole problem this release addresses.

What got quantized and what didn't

Only the routed MoE experts are quantized to NVFP4. Attention layers, router gates, shared experts, embeddings, and DeepSeek's DSpark speculative-decoding heads all remain in their original precision. The routed experts are where almost all parameter mass lives in a sparse MoE, so quantizing them slashes on-disk and VRAM footprint substantially while leaving the numerically sensitive paths untouched.

Why NVFP4 instead of MXFP4 or INT4

NVFP4 and MXFP4 both use symmetric per-group quantization with a fixed granularity, unlike INT4. MXFP4 uses a group size of 32 with an FP8 (E8M0) scaling factor per group; NVFP4 tightens that to a group size of 16 with an FP8 (E4M3) scaling factor, then adds a global FP32 scale per tensor to extend the representable range. Both scaling steps are hardware-accelerated on Blackwell.