Qwen releases an official 4-bit Qwen3-32B checkpoint

Alibaba's Qwen team has released Qwen3-32B-AWQ, an official 4-bit quantization of its 32.8-billion-parameter dense reasoning model. The checkpoint cuts raw weight storage from roughly 65.6GB in bf16 to about 20GB, allowing constrained deployments on 24GB GPUs such as the RTX 3090, RTX 4090, and L4. Qwen's benchmark comparison shows score changes ranging from a 2-point decline to a 0.6-point increase.

32B weights on a 24GB card

Qwen3-32B is the largest dense model in the original Qwen3 family, alongside mixture-of-experts models such as 30B-A3B and 235B-A22B. Dense models activate all their layers for each token, while those MoE variants route each token through a smaller subset of parameters. Qwen3-32B contains 32.8 billion total parameters, including 31.2 billion non-embedding parameters, across 64 layers. Its grouped-query attention uses 64 query heads and eight key-value heads.

Activation-aware Weight Quantization assigns 4-bit values to most linear-layer weights while using calibration data to preserve channels that strongly affect the output. Activations and the key-value cache remain at the precision selected by the inference engine, so total runtime memory exceeds the checkpoint's file size. CUDA kernels, temporary buffers, prompt length, batch size, and concurrency determine whether a deployment fits. A 24GB card leaves limited space for the key-value cache, especially with long contexts.

The model supports 32,768 tokens natively and up to 131,072 tokens through YaRN, a positional-scaling method that extends the usable context window. Major open-source engines generally apply a static YaRN scaling factor, which can reduce quality on shorter inputs. Qwen recommends enabling the extension only for workloads that need it. Key-value cache memory also grows with sequence length, making the full 128K window impractical on many single-card configurations.

One checkpoint, two decoding paths