---
title: "NVIDIA Shrinks Qwen3.5-122B to Run on a Single Blackwell GPU"
subtitle: "NVIDIA released a pre-quantized NVFP4 build of Qwen3.5-122B-A10B that shrinks the MoE model roughly 4x with near-zero accuracy loss on Blackwell."
url: "https://alphasignal.ai/news/nvidia-shrinks-qwen3-5-122b-to-run-on-a-single-blackwell-gpu"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "NVIDIA AI"
company_url: "https://alphasignal.ai/company/nvidiaai"
published: "2026-05-13T21:13:54.000Z"
updated: "2026-09-03T22:30:12.089Z"
type: "Model"
topics:
  - "Llms"
  - "Gpus"
subtopics:
  - "Mixture Of Experts"
  - "Long Context"
  - "Quantization"
upvotes: 3668
source_url: "https://huggingface.co/nvidia/Qwen3.5-122B-A10B-NVFP4"
comments: 0
truncated: true
---

# NVIDIA Shrinks Qwen3.5-122B to Run on a Single Blackwell GPU

> NVIDIA released a pre-quantized NVFP4 build of Qwen3.5-122B-A10B that shrinks the MoE model roughly 4x with near-zero accuracy loss on Blackwell.

NVIDIA has published a pre-quantized [NVFP4 checkpoint of Alibaba's Qwen3.5-122B-A10B](https://huggingface.co/nvidia/Qwen3.5-122B-A10B-NVFP4), a Mixture-of-Experts model with 122B total parameters and roughly 10B activated per token. The release targets Blackwell owners who want a 4-bit checkpoint they can serve directly through vLLM without running their own calibration pipeline. It has crossed 900K downloads on Hugging Face.

## How NVFP4 works

NVFP4 is a 4-bit floating-point format tied to [Blackwell tensor cores](https://build.nvidia.com/spark/nvfp4-quantization). Each value is stored in E2M1 (one sign bit, two exponent bits, one mantissa bit) with symmetric per-group quantization at a group size of 16, finer than MXFP4's group size of 32\. Each 16-value block stores one 4-bit number per element, one shared FP8 scaling factor, and one FP32 per-tensor scaling factor, averaging around 4.5 bits per value. The two-level scaling preserves dynamic range in weights that mix very large and very small values, which is where earlier FP4 schemes tended to fall apart.

Compared to FP16, memory footprint drops by roughly 3.5x; compared to FP8, by about 1.8x. Both figures are higher overhead than typical INT4 at block size 128, but NVFP4 earns it back in hardware: Blackwell tensor cores run matmuls directly in FP4, so there's no mid-inference dequantization step to bloat latency. INT4 inference requires unpacking weights back to 16-bit before computation; NVFP4 skips that entirely.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [nvidia/Qwen3.5-122B-A10B-NVFP4 model card](https://huggingface.co/nvidia/Qwen3.5-122B-A10B-NVFP4)
- [NVFP4: Same Accuracy with 2.3x Higher Throughput for 4-Bit LLMs](https://medium.com/data-science-collective/nvfp4-same-accuracy-with-2-3x-higher-throughput-for-4-bit-llms-03518ecba108)
- [NVFP4 vs MXFP4 Decision Guide](https://www.spheron.network/blog/nvfp4-vs-mxfp4-gpu-cloud-4bit-quantization-guide/)
- [A Comprehensive Evaluation on Quantization Techniques for LLMs](https://arxiv.org/pdf/2507.17417)
- [Quantize Models to NVFP4 with NVIDIA Model Optimizer](https://build.nvidia.com/spark/nvfp4-quantization)