Skip to main content
Back

NVIDIA Shrinks Qwen3.8-27B by 2.8x With Almost No Quality Loss

NVIDIA shipped a 4-bit quantized Qwen3.8-27B that cuts memory 2.8x while keeping BF16-level scores on reasoning, coding, and vision benchmarks.

Pro

  • NVIDIA released Qwen3.8-27B-NVFP4-QAD, a 4-bit distilled version of Qwen3.8-27B with 2.8x smaller memory footprint.
  • Benchmarks stay within 0.6 points of BF16 across GPQA, MMMU Pro, Terminal-Bench, SciCode, IFBench, AA-LCR.
  • Uses Quantization-Aware Distillation: KL divergence against a frozen BF16 teacher instead of cross-entropy retraining.
  • Supports text, image, video input and up to 262,144 token context length.
  • Requires NVIDIA Blackwell GPUs, Linux, and either vLLM or SGLang to serve.
  • Produced with NVIDIA Model Optimizer v0.48.0 under the NVIDIA Open Model License.

NVIDIA has published a 4-bit version of Alibaba’s Qwen3.8-27B on Hugging Face. The checkpoint, Qwen3.8-27B-NVFP4-QAD, uses quantization-aware distillation, or QAD, to preserve the BF16 model’s behavior while reducing the disk and GPU memory required by quantized components by approximately 2.8 times.

For developers, the release provides a pre-quantized multimodal model intended for single-GPU deployment on suitable NVIDIA Blackwell hardware. The quantized components use about 36% as much memory as their BF16 equivalents, although total serving memory also depends on the KV cache, context length, concurrency, and runtime overhead.

Near-BF16 scores at 4 bits

NVIDIA reports small differences between the BF16 teacher and NVFP4 student across science, instruction-following, multimodal, coding, agent, and long-context evaluations. The largest absolute change in the published table is 0.56 points.

NVIDIA-reported benchmark results. Delta equals NVFP4 QAD minus BF16.
Benchmark Capability BF16 NVFP4 QAD Delta
GPQA Diamond Graduate-level science 88.19 87.94 -0.25
IFBench Instruction following 79.25 79.04 -0.21
MMMU Pro Multimodal reasoning 75.78 75.61 -0.17
SciCode Scientific coding 48.34 48.82 +0.48
Terminal-Bench 2.1 Terminal agents 77.25 76.69 -0.56
AA-LCR Long-context recall 78.63 79.13 +0.50

Across the six tests, the quantized checkpoint trails on four and leads on two. The gains on SciCode and AA-LCR are small and do not establish that quantization improves quality. NVIDIA’s published figures show close point estimates, but they include no confidence intervals or multi-run variance.

Teacher logits preserve behavior

Post-training quantization calibrates scales and maps trained weights into a lower-precision format without optimizing them against a teacher. Conventional quantization-aware training exposes a model to simulated low-precision arithmetic during retraining, often using the original task loss. Recreating that process for a heavily post-trained model can require supervised fine-tuning data, reinforcement-learning infrastructure, reward models, and model-merging stages.