Alibaba's Qwen3.8-Flash-Next Hits 10x Long-Context Speed at 1/9 Training Cost

Alibaba's Qwen team has dropped an early architectural preview of what will become Qwen4, packaged as an open-weight model called Qwen3.8-Flash-Next. It is a multimodal Mixture-of-Experts system that swaps out most of the standard transformer attention stack for a new hybrid design, and the numbers on cost and long-context throughput are the kind of jump that tends to reshape how people build agents.

Qwen is publishing the architectural changes ahead of the full Qwen4 family so the community can evaluate them independently, the same role Qwen3-Next played for Qwen3.5. If GDN plus QSA holds up under scrutiny, expect it everywhere in the next generation.

The shape of the model

Qwen3.8-Flash-Next is a multimodal, ultra-sparse MoE with 125B parameters plus an additional 51B N-gram embedding table, while activating only 6B parameters per token. It accepts text and images, totals 176B parameters counting the N-gram table, and cuts both training and inference cost substantially compared to Qwen3.7-Plus (training takes roughly 1/9 as much) while holding comparable overall quality. The hosted production variant is served as qwen3.8-flash on QwenCloud.

Pricing on the hosted version lands at $0.16 per million input tokens and $0.47 per million output tokens, putting it firmly in the ultra-cheap-inference tier alongside DeepSeek and Gemini Flash. Native context is 262K tokens, extensible to 1M with YaRN.

Four architectural bets

The technical report frames the release around four upgrades, each aimed at a different bottleneck in scaling long-context, multi-tool agent workloads.

• Hybrid attention (GDN + QSA). Three of every four layers use Gated DeltaNet to compress history; the fourth uses Qwen Sparse Attention for precise long-range retrieval. GDN is a linear-attention layer that maintains a fixed-size recurrent state and updates it with a learned gating rule, with a lineage running through DeltaNet and Mamba-2. QSA is the new piece: rather than selecting individual tokens for processing, it operates at the micro-block level, which cuts long-context latency significantly.
• Gated Residual. Residual streams with normalization are what make deep LLM training manageable. Gated Residual modulates information flowing through widened residual streams via an element-wise, data-dependent read gate and a per-branch scalar write gate. Practically, this widens the residual highway to four branches with dynamic gating, which the team credits for stronger cross-layer information flow and better training stability.
• N-gram embedding. The 51B parameter table looks huge but is compute-free at inference. The lookup memory adds capacity with little per-token compute and can be asynchronously offloaded to host memory. The embeddings use deterministic lookups, so they add zero matrix-multiplication cost per token.
• Muon optimizer. Muon handles the genuine 2-D linear maps (attention, GDN and MoE expert weights), while AdamW handles embeddings, the MoE router and GR's low-rank parameters. Fused QKV / SwiGLU / GDN projections are split before orthogonalization, the scaling law was refit for the new architecture, and batch-size warmup was dropped after it cost 18.8% more optimizer steps without improving the result.

The long-context payoff

Speed at long contexts is where the numbers get loud. Qwen reports that QSA reaches up to 10.2x prefill and 6.6x decode attention-kernel speedups at one million tokens. The team's own tweet cites a more conservative 7.6x prefill and 4.9x decode at 1M, and claims 8.6x the end-to-end prefill throughput of Qwen3.7-Plus when a 90% prefix-cache hit rate is achieved.

A built-in Multi-Token Prediction module supports speculative decoding, and the MTP module's own attention layers are QSA, which keeps speculative acceptance high in practice.

Benchmarks

With only 6B active parameters, the base model matches or beats Qwen3.7-Plus on most language benchmarks. The instruct model is aimed squarely at coding and agentic office work:

Benchmark	Score	What it measures
DeepSWE 1.1	58.7	Autonomous software engineering
SWE-bench Pro	62.5	Real repo bug-fixing
CoWorkBench	73.9	Multi-turn office/collaboration tasks
AndroidWorld	84.5	Mobile UI agents
MathVision (with CI)	95.7	Visual math reasoning

Running it yourself

Both vLLM and SGLang have day-zero support. The FP8 checkpoint fits in an 8x GPU node with tensor and expert parallelism, and a rough vLLM launch looks like this:

vllm serve Qwen/Qwen3.8-Flash-Next-FP8 \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --moe-backend triton \
  --enable-prefix-caching \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3

One catch worth knowing: plain TP8 is incompatible with the FP8 checkpoint, so you need TEP8 (tensor plus expert parallelism). On AMD, ROCm builds run with AITER enabled but MoE-AITER off. N-gram embedding offload currently only runs on NVIDIA devices.

Why the ratio matters

The 6B-active, 125B-total ratio is aggressive even by 2026 MoE standards, and combined with the N-gram lookup table, it points toward a workflow where memory bandwidth matters more than FLOPs. That has real implications for hardware. As one early observer put it, this is the kind of architecture that could get interesting on 128GB Strix Halo boxes, 128GB+ Apple Silicon, DGX Spark and large multi-GPU rigs, because you get big-model capacity with only around 6B of neural parameters firing per token.

The framing is worth noticing too. Qwen has now standardized on releasing a Flash-Next model as an architectural probe before each major generation, publishing it under open weights so the community can stress-test the design before the flagship arrives. Everything about this release, from the linear-attention majority to the sparse indexer to the N-gram memory offload, is a bet that the next bottleneck for frontier models is the cost of shoving a million tokens through attention every few seconds, rather than raw parameter count. If the benchmarks hold up under independent evaluation, that bet looks good.