Qwen3.8-Flash-Next fits in a 75.2 GB GGUF

Community quantizer PeasantSmith has published a mixed-precision version of Alibaba’s Qwen3.8-Flash-Next. The release packages the roughly 177 billion language-model parameters into one 75.2 GB GGUF, with a separate 0.9 GiB vision projector, and had passed 2,600 downloads at the time of writing. Its component-specific precision preserves sensitive weights while compressing routed experts aggressively.

GGUF packages model tensors and metadata for local runtimes such as llama.cpp. This build gives developers a workstation-scale route to test a multimodal, long-context mixture-of-experts model whose full-precision weights would require substantially more memory.

177B total, 6B active

Qwen3.8-Flash-Next combines a 125B-parameter main model with 51B parameters of n-gram embeddings. Its mixture-of-experts architecture activates about 6B parameters for each token. The model accepts images and supports a native context window of 256K tokens.

Its 48 computational layers form 12 macro-blocks. Each block contains three Gated DeltaNet layers with MoE, followed by one Qwen Sparse Attention layer with MoE; four-branch Gated Residual connections wrap the block. Qwen Sparse Attention routes attention over micro-blocks, reducing the selection granularity and targeting lower latency at long context lengths. Alibaba presents the design as an early preview of the architecture planned for Qwen4, similar to the role Qwen3-Next served before Qwen3.5.

Where the precision goes

The build allocates precision according to each tensor’s role. A uniform 2-bit layout would expose sensitive embeddings, output weights, and residual-writing projections to the same error budget as the more compressible expert tensors. The 51.2B-parameter PLE n-gram table receives 4.5 bits per weight and can be placed in system RAM, preserving more information without consuming the entire GPU memory budget.