Ant Group's Ling 3.0 Flash Beats a 1T Model With 5B Active Parameters
Ant Group's Ling 3.0 Flash packs 124B parameters with only 5B active at inference, beating its own 1T flagship on most benchmarks at a fraction of the compute cost.
- Ant Group's inclusionAI released Ling 3.0 Flash, a 124B-parameter MoE reasoning model with only 5.1B active parameters per token.
- It scores 38 on the Artificial Analysis Intelligence Index, a 24-point jump over the previous generation, matching MiMo-V2.5 and Qwen3 27B with far fewer active parameters.
- Architecture uses a novel hybrid of KDA (linear attention) and MLA layers at a 5:1 ratio, plus a halved MoE expert activation ratio (1/64 vs. 1/32), baked in from pretraining.
- Weights are MIT-licensed and available on Hugging Face in BF16 (255GB) and FP8 (128GB); API access via inclusionAI and DeepInfra at $0.075/$0.22 per 1M tokens.
- Strong on agentic benchmarks (2nd among flash-tier open-weights on τ³-Banking), but very verbose: generates ~4x more output tokens than peers, raising cost-per-task.
- Hallucination rate dropped from 97% to 44% mostly by learning to abstain, not by gaining factual knowledge; raw accuracy only moved from 16% to 18%.
Ant Group's AI lab inclusionAI just released Ling 3.0 Flash, a 124B-parameter reasoning model that challenges a core assumption in the field: that you need a massive active parameter count to get top-tier intelligence. With only 5.1B parameters active per token at inference time, the model punches well above its weight class and, according to third-party evaluations, lands on the open-weights Pareto frontier for intelligence versus total parameters.
Ling 3.0 Flash is a hybrid-reasoning MoE model built for production-scale agents, with 124B total parameters and just 5.1B active per token. With 1/8 of the total and 1/12 of the active parameters, it matches or beats the previous 1T flagship model on most benchmarks. That is a striking result: the new model is dramatically cheaper to run while delivering better outputs.
A New Architecture, Not Just a Smaller Model
The efficiency gains are not from simple pruning or distillation. Ling 3.0 Flash moves away from the traditional approach of simply scaling parameter counts, and is instead built from the ground up with a native hybrid-linear attention architecture, alternating KDA (Kimi Delta Attention) and MLA layers at a 5:1 ratio to optimally balance long-context efficiency with robust state memory.
To understand why this matters: standard Transformer attention has a cost that grows quadratically with sequence length. Linear attention (like KDA) replaces this with a recurrent state that keeps cost constant regardless of context length, but historically at the expense of recall quality. By mixing KDA with MLA (Multi-head Latent Attention, a more standard but memory-compressed attention mechanism) at a 5:1 ratio, Ling 3.0 gets the speed of linear attention for most layers while preserving precise recall where it counts.
Three architectural upgrades drive the efficiency leap:
- Upgraded KDA: Evolving from the previous Lightning Attention, KDA introduces fine-grained diagonal gating in Delta Rule state updates, allowing the model to retain critical information more precisely when processing lengthy documents and extensive codebases.