Skip to main content
Back

NVIDIA Shrinks Alibaba's Qwen3.6 35B by 3x Without Losing Accuracy

A 35B Mixture-of-Experts model squeezed into 4-bit floating point, shrinking memory roughly threefold while holding benchmark accuracy within a point of the original.

Model, Full Access
  • NVIDIA released an NVFP4-quantized Qwen3.6-35B-A3B on Hugging Face under Apache 2.0.
  • 35B total, 3B active MoE parameters with hybrid attention and 262K context length.
  • Memory shrinks roughly 3.06x versus BF16 while benchmark accuracy stays within a point.
  • Multimodal inputs: text, RGB images, and MP4/WebM video; text output only.
  • Runs on vLLM across Hopper and Blackwell, quantized with NVIDIA Model Optimizer v0.44.0.
  • NVFP4 uses 16-value blocks with E4M3 + FP32 two-level scaling, beating INT4 on dynamic range.

NVIDIA has dropped a pre-quantized build of Alibaba's Qwen3.6-35B-A3B onto Hugging Face, rebuilt in the company's NVFP4 4-bit floating-point format. The headline trade is simple: roughly a third of the memory, with accuracy still inside spitting distance of the BF16 baseline.

The base model is a Mixture-of-Experts (MoE) with hybrid attention, 35B total parameters and 3B activated per token. The quantized release was produced using NVIDIA's Model Optimizer toolkit, and the resulting checkpoint reports as a 19B-parameter footprint on disk, mixing BF16, FP8 E4M3, and U8 tensors.

What NVFP4 actually is

NVFP4 is NVIDIA's answer to the accuracy cliff that hits most 4-bit schemes. It is a 4-bit floating point format that uses a two-level scaling strategy with a fine-grained E4M3 scaling factor and a second-level FP32 scalar, applied over blocks of 16 values rather than the 32 used by its predecessor MXFP4. Smaller blocks mean the scale adapts more tightly to local dynamic range, which is where INT4 traditionally falls apart on transformer activations.

The E2M1 layout (1 sign bit, 2 exponent bits, 1 mantissa bit) lets the format represent values across a wider dynamic range than INT4, which matters because activation and weight distributions in transformers span many orders of magnitude, so NVFP4 tends to produce better output quality than INT4 at the same memory footprint. The catch is hardware: NVFP4 is a native tensor core format in Blackwell, accelerated by dedicated compute units that didn't exist in Hopper or earlier architectures.