Tencent Open-Sources AngelSpec, Hitting 2.86x Faster Inference on Hunyuan 3
Tencent open-sources AngelSpec, a full training framework for speculative decoding that delivers up to 2.40x speedup on its 295B Hunyuan 3 model
- AngelSpec open-sourced: Tencent releases a full training + deployment framework for speculative decoding draft models.
- Up to 2.86x speedup: DFly architecture achieves 1.98-2.40x average throughput gain over autoregressive decoding on Hunyuan 3 (295B MoE).
- Six draft architectures: DFly, DFlash, DFlare, Eagle3, DSpark, and MTP all supported via one pipeline with config-level switching.
- Disaggregated training: Inference and training GPU pools scale independently via RDMA-backed hidden-state streaming (no disk staging).
- Pre-trained weights released: Hugging Face models for Hy3 and Qwen3-8B available now with vLLM support.
- 128k context training: MTP path supports long-sequence training via Ulysses sequence parallelism.
Speculative decoding has become one of the most practical tools for squeezing more throughput out of large language models without touching the underlying weights. The idea is simple: a small, cheap draft model proposes several tokens at once, and the full target model verifies them all in a single forward pass. If the drafts are good enough, you get multiple tokens for the cost of one. The hard part is training a draft model that's fast, accurate, and works at production scale. That's exactly what Tencent's Hunyuan team just open-sourced.
AngelSpec is a torch-native, end-to-end framework for training speculative decoding draft models, covering both the training pipeline and the deployment integration. It's not a research prototype: the framework was built to serve Tencent's own Hunyuan 3 (Hy3) model in production, a 295B MoE with only ~21B active parameters per forward pass.
The numbers that matter
On Hy3-A21B, DFly delivers the highest throughput across all concurrency levels (4 to 64) and all six benchmarks, with a 1.98 to 2.40x average speedup over the autoregressive baseline, peaking at 2.86x on code and math tasks, and running 10.5 to 11.8% faster than DFlash. The gap in accepted token length is even more telling: DFly reaches a mean accepted length of 4.79, which is +30% over DFlash (3.69) and roughly 1.6x over MTP (3.00), going up to 5.52 on HumanEval.
Accepted length is the key metric here. It measures how many draft tokens the target model accepts on average before rejecting one. A higher number means fewer full forward passes, which translates directly to throughput.
Six draft architectures, one training pipeline
The most unusual thing about AngelSpec is its breadth. Rather than committing to a single drafting strategy, the framework supports six distinct architectures, all trained through the same pipeline with a config change: