---
title: "NVIDIA Shrinks Alibaba's Qwen3.5-397B to Run on Half the GPUs"
subtitle: "NVIDIA's FP4-quantized Qwen3.5-397B fits on 4 Blackwell GPUs instead of 8, with near-zero accuracy loss across every benchmark tested."
url: "https://alphasignal.ai/news/nvidia-shrinks-alibaba-s-qwen3-5-397b-to-run-on-half-the-gpus"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "NVIDIA AI"
company_url: "https://alphasignal.ai/company/nvidiaai"
published: "2026-06-30T02:46:57.000Z"
updated: "2026-07-09T02:47:11.603Z"
type: "Model"
topics:
  - "Gpus"
  - "Llms"
subtopics:
  - "Quantization"
  - "Mixture Of Experts"
upvotes: 2466
source_url: "https://huggingface.co/nvidia/Qwen3.5-397B-A17B-NVFP4-V2"
comments: 0
truncated: true
---

# NVIDIA Shrinks Alibaba's Qwen3.5-397B to Run on Half the GPUs

> NVIDIA's FP4-quantized Qwen3.5-397B fits on 4 Blackwell GPUs instead of 8, with near-zero accuracy loss across every benchmark tested.

NVIDIA just published a production-ready, FP4-quantized version of Alibaba's [Qwen3.5-397B-A17B](https://huggingface.co/nvidia/Qwen3.5-397B-A17B-NVFP4-V2) on Hugging Face. The headline number is straightforward: a model with 397 billion parameters, compressed so aggressively that it now fits on half the hardware it previously required, while losing almost nothing in quality. That combination is what makes this release worth paying attention to.

## The model underneath

Qwen3.5-397B-A17B is a 397 billion parameter multimodal vision-language model from Alibaba's Qwen team, featuring a hybrid Gated DeltaNet and Mixture-of-Experts (MoE) architecture. The MoE part means only a fraction of the model is active at any moment. With only 17 billion parameters active per forward pass, 23% fewer than the prior 235B version despite having 69% more total parameters, it achieves comparable or better performance at lower inference cost. 

The architectural bet here is the hybrid attention design. Qwen3.5 builds on the Qwen3-Next lineage, combining Gated Delta Networks (a linear attention variant) with sparse Mixture-of-Experts. The model alternates between Gated DeltaNet layers and full attention layers in roughly a 3:1 ratio. Gated DeltaNet is a form of linear attention that writes to a fixed-size memory state instead of computing a full token-to-token attention matrix, which means the fixed-size recurrent state of Gated DeltaNet eliminates the linearly growing KV cache and gives near-constant per-token decode cost. The periodic full-attention layers are kept because DeltaNet is less exact at content-based retrieval. 

The model has 512 experts (4x more than Qwen3's 128) with 10+1 active experts per token, and was trained with multi-token prediction, enabling speculative decoding for 2-3x inference speedup. Qwen3.5-397B-A17B is in the same performance tier as Gemini 3 Pro, Claude Opus 4.5, and GPT-5.2\. 

## What NVFP4 actually does

NVFP4 (NVIDIA FP4) is a 4-bit floating-point format native to Blackwell architecture. It uses the E2M1 format (2-bit exponent, 1-bit mantissa) with blockwise quantization and FP8 scaling factors. To put that in plain terms: each weight is stored using just 4 bits instead of the usual 16, but the scaling is done carefully at the block level to preserve as much numerical range as possible.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [nvidia/Qwen3.5-397B-A17B-NVFP4-V2 · Hugging Face](https://huggingface.co/nvidia/Qwen3.5-397B-A17B-NVFP4-V2)
- [NVFP4 Quantization Guide – local-inference-lab](https://github.com/local-inference-lab/rtx6kpro/blob/master/optimization/nvfp4-quantization.md)
- [NVIDIA Blackwell Delivers Massive Performance Leaps in MLPerf Inference v5.0](https://developer.nvidia.com/blog/nvidia-blackwell-delivers-massive-performance-leaps-in-mlperf-inference-v5-0/)
- [Qwen3.5: Nobody Agrees on Attention Anymore – Hugging Face Blog](https://huggingface.co/blog/mlabonne/qwen35)
- [Creating the NVIDIA Nemotron 3 Ultra NVFP4 Checkpoint with NVIDIA Model Optimizer](https://developer.nvidia.com/blog/creating-the-nvidia-nemotron-3-ultra-nvfp4-checkpoint-with-nvidia-model-optimizer/)