---
title: "NVIDIA Shrinks Alibaba's Qwen3.6 35B by 3x Without Losing Accuracy"
subtitle: "A 35B Mixture-of-Experts model squeezed into 4-bit floating point, shrinking memory roughly threefold while holding benchmark accuracy within a point of the original."
url: "https://alphasignal.ai/news/nvidia-shrinks-alibaba-s-qwen3-6-35b-by-3x-without-losing-accuracy"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "NVIDIA AI"
company_url: "https://alphasignal.ai/company/nvidiaai"
published: "2026-05-30T12:00:00.000Z"
updated: "2026-06-04T22:58:04.528Z"
type: "Model"
topics:
  - "Gpus"
  - "Llms"
subtopics:
  - "Quantization"
  - "Mixture Of Experts"
  - "Long Context"
upvotes: 108
source_url: "https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4"
comments: 0
truncated: true
---

# NVIDIA Shrinks Alibaba's Qwen3.6 35B by 3x Without Losing Accuracy

> A 35B Mixture-of-Experts model squeezed into 4-bit floating point, shrinking memory roughly threefold while holding benchmark accuracy within a point of the original.

NVIDIA has dropped a pre-quantized build of Alibaba's Qwen3.6-35B-A3B onto Hugging Face, rebuilt in the company's [NVFP4](https://developer.nvidia.com/blog/introducing-nvfp4-for-efficient-and-accurate-low-precision-inference/) 4-bit floating-point format. The headline trade is simple: roughly a third of the memory, with accuracy still inside spitting distance of the BF16 baseline.

The base model is a Mixture-of-Experts (MoE) with hybrid attention, 35B total parameters and 3B activated per token. The quantized release was produced using NVIDIA's Model Optimizer toolkit, and the resulting checkpoint reports as a 19B-parameter footprint on disk, mixing BF16, FP8 E4M3, and U8 tensors.

## What NVFP4 actually is

NVFP4 is NVIDIA's answer to the accuracy cliff that hits most 4-bit schemes. It is a 4-bit floating point format that uses a two-level scaling strategy with a fine-grained E4M3 scaling factor and a second-level FP32 scalar, applied over blocks of 16 values rather than the 32 used by its predecessor MXFP4\. Smaller blocks mean the scale adapts more tightly to local dynamic range, which is where INT4 traditionally falls apart on transformer activations.

The E2M1 layout (1 sign bit, 2 exponent bits, 1 mantissa bit) lets the format represent values across a wider dynamic range than INT4, which matters because activation and weight distributions in transformers span many orders of magnitude, so NVFP4 tends to produce better output quality than INT4 at the same memory footprint. The catch is hardware: NVFP4 is a native tensor core format in Blackwell, accelerated by dedicated compute units that didn't exist in Hopper or earlier architectures.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [nvidia/Qwen3.6-35B-A3B-NVFP4 Model Card](https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4)
- [Introducing NVFP4 for Efficient and Accurate Low-Precision Inference](https://developer.nvidia.com/blog/introducing-nvfp4-for-efficient-and-accurate-low-precision-inference/)
- [FP4 Quantization on Blackwell GPUs: Throughput, Cost, and When It's Worth It](https://www.spheron.network/blog/fp4-quantization-blackwell-gpu-cost/)
- [NVFP4: Same Accuracy with 2.3x Higher Throughput for 4-Bit LLMs](https://medium.com/data-science-collective/nvfp4-same-accuracy-with-2-3x-higher-throughput-for-4-bit-llms-03518ecba108)
- [NVIDIA Blackwell; The Impact of NVFP4 For LLM Inference](https://www.nota.ai/community/where-do-llms-encode-the-knowledge-to-assess-the-ambiguity-6zjbl-tmjns-l25lf-grh8b)