Qwen3.8 Flash Next gains 256K vision on dual V100s

Version 3 of the community-built project model card adds image input across the model’s 262,144-token native context and fixes a fused CUDA kernel race that caused NaN scores, crashes, and higher perplexity. The package runs Alibaba’s sparse Mixture-of-Experts model on two Nvidia Tesla V100 32 GB GPUs through an 80 GiB GGUF file and a patched llama.cpp runtime.

The hardware target makes the release useful for labs with V100 servers or older DGX systems. Most current inference stacks have dropped optimized support for Volta GPUs, yet this build reports about 120 tokens per second for short-context decoding and 112.8 tokens per second with a 250K KV cache.

The architecture behind the build

Alibaba describes Qwen architecture post as an early preview of the design intended for Qwen4. Qwen3.8 Flash Next contains a 125B-parameter main model plus 51B parameters of N-gram embeddings, while its sparse expert routing activates about 6B parameters for each token. Alibaba reports training costs around one-ninth of Qwen3.7 Plus.

The architecture combines recurrent-style Gated DeltaNet layers with Qwen Sparse Attention, or QSA, which reduces attention work over long sequences. Native context reaches 262,144 tokens, and Alibaba says YaRN context scaling can extend that limit to one million tokens.

Inside the 80 GiB GGUF

The build uses a single-file GGUF, a format that packages model tensors and metadata for local inference. Its quantization mix keeps the routed experts within the two GPUs’ combined 64 GB of VRAM and restores the model’s multi-token-prediction head for speculative decoding.

• Routed experts: