NVIDIA Shrinks Qwen3.8-27B by 2.8x With Almost No Quality Loss
NVIDIA shipped a 4-bit quantized Qwen3.8-27B that cuts memory 2.8x while keeping BF16-level scores on reasoning, coding, and vision benchmarks.
Takeaways
- NVIDIA released Qwen3.8-27B-NVFP4-QAD, a 4-bit distilled version of Qwen3.8-27B with 2.8x smaller memory footprint.
- Benchmarks stay within 0.6 points of BF16 across GPQA, MMMU Pro, Terminal-Bench, SciCode, IFBench, AA-LCR.
- Uses Quantization-Aware Distillation: KL divergence against a frozen BF16 teacher instead of cross-entropy retraining.
- Supports text, image, video input and up to 262,144 token context length.
- Requires NVIDIA Blackwell GPUs, Linux, and either vLLM or SGLang to serve.
- Produced with NVIDIA Model Optimizer v0.48.0 under the NVIDIA Open Model License.
NVIDIA has published a 4-bit version of Alibaba’s Qwen3.8-27B on Hugging Face. The checkpoint, Qwen3.8-27B-NVFP4-QAD, uses quantization-aware distillation, or QAD, to preserve the BF16 model’s behavior while reducing the disk and GPU memory required by quantized components by approximately 2.8 times.
For developers, the release provides a pre-quantized multimodal model intended for single-GPU deployment on suitable NVIDIA Blackwell hardware. The quantized components use about 36% as much memory as their BF16 equivalents, although total serving memory also depends on the KV cache, context length, concurrency, and runtime overhead.
Near-BF16 scores at 4 bits
NVIDIA reports small differences between the BF16 teacher and NVFP4 student across science, instruction-following, multimodal, coding, agent, and long-context evaluations. The largest absolute change in the published table is 0.56 points.
| Benchmark | Capability | BF16 | NVFP4 QAD | Delta |
|---|---|---|---|---|
| GPQA Diamond | Graduate-level science | 88.19 | 87.94 | -0.25 |
| IFBench | Instruction following | 79.25 | 79.04 | -0.21 |
| MMMU Pro | Multimodal reasoning | 75.78 | 75.61 | -0.17 |
| SciCode | Scientific coding | 48.34 | 48.82 | +0.48 |
| Terminal-Bench 2.1 | Terminal agents | 77.25 | 76.69 | -0.56 |
| AA-LCR | Long-context recall | 78.63 | 79.13 | +0.50 |
Across the six tests, the quantized checkpoint trails on four and leads on two. The gains on SciCode and AA-LCR are small and do not establish that quantization improves quality. NVIDIA’s published figures show close point estimates, but they include no confidence intervals or multi-run variance.
Teacher logits preserve behavior
Post-training quantization calibrates scales and maps trained weights into a lower-precision format without optimizing them against a teacher. Conventional quantization-aware training exposes a model to simulated low-precision arithmetic during retraining, often using the original task loss. Recreating that process for a heavily post-trained model can require supervised fine-tuning data, reinforcement-learning infrastructure, reward models, and model-merging stages.