Tencent Open-Sources AngelSpec, Hitting 2.86x Faster Inference on Hunyuan 3
Tencent open-sources AngelSpec, a full training framework for speculative decoding that delivers up to 2.40x speedup on its 295B Hunyuan 3 model
- AngelSpec open-sourced: Tencent releases a full training + deployment framework for speculative decoding draft models.
- Up to 2.86x speedup: DFly architecture achieves 1.98-2.40x average throughput gain over autoregressive decoding on Hunyuan 3 (295B MoE).
- Six draft architectures: DFly, DFlash, DFlare, Eagle3, DSpark, and MTP all supported via one pipeline with config-level switching.
- Disaggregated training: Inference and training GPU pools scale independently via RDMA-backed hidden-state streaming (no disk staging).
- Pre-trained weights released: Hugging Face models for Hy3 and Qwen3-8B available now with vLLM support.
- 128k context training: MTP path supports long-sequence training via Ulysses sequence parallelism.
Speculative decoding works by having a small, cheap draft model propose several tokens at once while the full target model verifies them all in a single forward pass. When the drafts are accurate, you get multiple tokens for the cost of one. The hard part is training a draft model that's fast, accurate, and holds up at production scale. Tencent's Hunyuan team just open-sourced a framework built to do exactly that.
AngelSpec is a torch-native, end-to-end framework for training speculative decoding draft models, covering both the training pipeline and deployment integration. Built to serve Tencent's own Hunyuan 3 (Hy3) model in production, a 295B MoE with roughly 21B active parameters per forward pass, it shipped as a production system before it shipped as open source.
The numbers that matter
On Hy3-A21B, the top-performing architecture (DFly) delivers a 1.98 to 2.40x average speedup over the autoregressive baseline across all concurrency levels (4 to 64) and all six benchmarks, peaking at 2.86x on code and math tasks. It runs 10.5 to 11.8% faster than DFlash, the next best option.
The more revealing metric is mean accepted token length, which measures how many draft tokens the target model accepts before rejecting one. Fewer rejections mean fewer full forward passes, which translates directly to throughput. DFly reaches a mean accepted length of 4.79, about 30% above DFlash (3.69) and 60% above MTP (3.00), climbing to 5.52 on HumanEval.