Alibaba's Qwen3.8-Flash-Next Shrinks 125B Model to 84 GiB on One Machine
A community quant of Qwen3.8-Flash-Next shrinks the 177B MoE to 84 GiB with a per-layer mixed-precision recipe that beats standard IQ4_XS on both size and quality.
PRO- agentionai released Agention Precision GGUF quants of Qwen3.8-Flash-Next, mainline llama.cpp compatible.
- AP-IQ4_XS fits in 84.24 GiB versus 117.13 GiB for AesSedai's IQ4_XS at similar quality.
- AP-Q4_K_XL reaches perplexity within 1.17% of the unquantized reference on wikitext-2.
- Base model is a 125B MoE with 6B active parameters plus a 51B N-gram embedding table.
- Architecture support landed via llama.cpp PR 27742 from Daniel Han for qwen4_exp.
- Works out of the box with llama.cpp, Ollama, LM Studio, vLLM, and Docker Model Runner.
A new community release on Hugging Face squeezes Alibaba's Qwen3.8-Flash-Next into a form that fits on a single high-end workstation without sacrificing much quality. The Qwen3.8-Flash-Next-AP-GGUF repo from agentionai ships what it calls Agention Precision (AP) quants: mainline llama.cpp-compatible GGUFs that compress the model to as little as 84 GiB while keeping perplexity within a few percent of the original.
Qwen3.8-Flash-Next is a multimodal, ultra-sparse Mixture-of-Experts model with 125B parameters, including a 51B N-gram embedding table, activating just 6B parameters per token. An experimental preview of the architecture that will underpin Qwen4, it arranges 48 layers as 12 macro-blocks, each following a pattern of three Gated DeltaNet and MoE pairs followed by one Qwen Sparse Attention and MoE pair, all wrapped in a 4-branch Gated Residual. The main backbone runs 512 experts with 10 routed plus one shared per token. Add the 51B N-gram table (20M bigram and trigram entries, offloadable to host RAM) and a 4B MTP head, and the model stores roughly 180B parameters while computing only about 6B per token.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
