Ant Group's Ling-3.1-flash Squeezes a 560B Model Into Four GPUs
Ant Group's InclusionAI opens weights for a 560B hybrid-linear MoE model with 25B active parameters, 1M context, and MIT licensing.
- InclusionAI released Ling-3.1-flash, a 560B total / 25B active MoE under MIT license.
- Hybrid-linear attention: 63 KDA layers plus 9 gated MLA layers in a 7:1 ratio across 72 total.
- Supports up to 1M token context; 262K served on gateways with 32K max output.
- MXFP4 experts plus block-FP8 dense layers shrink weights from 534GB to 292GB, enabling 4-GPU Hopper serving.
- Scores 41 on Artificial Analysis Intelligence Index at ~210 output tokens/second.
- Day-0 support in SGLang and vLLM with MTP speculative decoding.
Ling-3.1-flash brings a 560B MoE model to four Hopper GPUs
Ant Group’s InclusionAI lab has released Ling-3.1-flash on Hugging Face under the MIT license. The mixture-of-experts model contains 560 billion parameters, activates 25 billion for each token, and supports sequences of up to 1 million tokens. It is among the largest openly licensed MoE releases and the first Ling 3.1 model with downloadable weights.
Routing only 25 billion parameters reduces the computation required for each token. Standard GPU serving still needs access to the full 560 billion parameter pool, so quantization, memory capacity, and cache growth determine whether the model is practical to deploy.
560B stored, 25B active
| Component | Specification |
|---|---|
| Total parameters | 560 billion |
| Active parameters | 25 billion per token |
| Layer mix | 63 KDA layers and 9 gated MLA layers |
| Feed-forward design | 4 dense layers, 512 routed experts, and 1 shared expert |
| Expert routing | 8 routed experts activated per token |
| Model dimensions | 4,096 hidden size and 64 attention heads |
| Vocabulary | 157,184 tokens |
| Maximum context | 1 million tokens |
| Published formats | Official FP8 weights and a mixed MXFP4/block-FP8 deployment recipe |
The architecture, called BailingMoeV3, combines sparse expert routing with a seven-to-one mix of linear and full attention. Four dense feed-forward layers handle shared computation. The sparse layers select eight of 512 routed experts for each token and also use one shared expert.
Kimi Delta Attention, or KDA, updates a fixed-size recurrent state as tokens arrive. Its diagonal gates control which information is retained or overwritten, allowing attention work to scale linearly with sequence length. Every eighth layer uses gated Multi-Head Latent Attention, which performs full attention over earlier positions through a compressed key-value cache.
Linear attention trims the cache
The 63 KDA layers maintain fixed-size state, leaving only nine MLA layers with a token-dependent cache. InclusionAI estimates approximately 0.5 GB of fixed state per conversation plus 10 KB for each token. A conversation at the advertised 1 million-token limit would therefore consume about 10.5 GB of cache before accounting for runtime workspaces and other serving overhead.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
