Unsloth just released NVFP4-quantized versions of Qwen3.6 -- both the 27B dense model and the 35B-A3B MoE -- with a headline claim of 2.5x faster inference on NVIDIA GPUs. The 35B-A3B variant hits 17,561 tokens/second on a B200. That's not a typo. And the 27B now fits in 24GB of VRAM, which means a single consumer or prosumer GPU can run it.
What NVFP4 actually is
To understand why this matters, you need to know what NVFP4 is. Most quantization formats (INT4, GGUF Q4) store weights in 4-bit integers but still have to dequantize back to 16-bit floats before any computation happens. That dequantization step is a hidden tax on every forward pass.
NVFP4 is NVIDIA's 4-bit floating-point format for Blackwell GPUs. It uses two-level scaling -- an FP8 micro-block scale per 16 values plus an FP32 tensor-level scale -- achieving roughly 1.8x memory reduction versus FP8. More importantly, because Blackwell Tensor Cores are designed to handle NVFP4 operations directly, tensors stay in their compact 4-bit format throughout inference. There is no dequantization, and NVFP4 operations are hardware-accelerated.
In plain terms: INT4 pretends to be 4-bit but secretly does 16-bit math. NVFP4 on a Blackwell GPU actually does 4-bit math end-to-end. Compared to FP8, NVFP4 offers 2-3x higher arithmetic throughput and approximately 1.8x memory reduction.
What Unsloth added on top
The raw NVFP4 format is not new -- NVIDIA introduced it with the Blackwell architecture. What Unsloth brings is their calibration and quantization pipeline applied to it, plus a key integration: MTP tensors are directly integrated inside the NVFP4 quant, and both vLLM and SGLang work for this.
Don't miss what's next in AI
Join 300,000+ engineers and researchers who get the signal, not the noise.
- Full access to in-depth AI research breakdowns
- Be the first to know what's trending before it hits mainstream
- Daily curated papers, repos, and industry moves