Alibaba's Qwen3.8-Flash-Next Shrinks 125B Model to 84 GiB on One Machine
A community quant of Qwen3.8-Flash-Next shrinks the 177B MoE to 84 GiB with a per-layer mixed-precision recipe that beats standard IQ4_XS on both size and quality.
Takeaways
- agentionai released Agention Precision GGUF quants of Qwen3.8-Flash-Next, mainline llama.cpp compatible.
- AP-IQ4_XS fits in 84.24 GiB versus 117.13 GiB for AesSedai's IQ4_XS at similar quality.
- AP-Q4_K_XL reaches perplexity within 1.17% of the unquantized reference on wikitext-2.
- Base model is a 125B MoE with 6B active parameters plus a 51B N-gram embedding table.
- Architecture support landed via llama.cpp PR 27742 from Daniel Han for qwen4_exp.
- Works out of the box with llama.cpp, Ollama, LM Studio, vLLM, and Docker Model Runner.
A new community release on Hugging Face squeezes Alibaba's Qwen3.8-Flash-Next into a form that fits on a single high-end workstation without sacrificing much quality. The Qwen3.8-Flash-Next-AP-GGUF repo from agentionai ships what it calls Agention Precision (AP) quants: mainline llama.cpp-compatible GGUFs that compress the model to as little as 84 GiB while keeping perplexity within a few percent of the original.
Qwen3.8-Flash-Next is a multimodal, ultra-sparse Mixture-of-Experts model with 125B parameters, including a 51B N-gram embedding table, activating just 6B parameters per token. An experimental preview of the architecture that will underpin Qwen4, it arranges 48 layers as 12 macro-blocks, each following a pattern of three Gated DeltaNet and MoE pairs followed by one Qwen Sparse Attention and MoE pair, all wrapped in a 4-branch Gated Residual. The main backbone runs 512 experts with 10 routed plus one shared per token. Add the 51B N-gram table (20M bigram and trigram entries, offloadable to host RAM) and a 4B MTP head, and the model stores roughly 180B parameters while computing only about 6B per token.