Tencent Open-Sources AngelSpec, Hitting 2.86x Faster Inference on Hunyuan 3

Speculative decoding works by having a small, cheap draft model propose several tokens at once while the full target model verifies them all in a single forward pass. When the drafts are accurate, you get multiple tokens for the cost of one. The hard part is training a draft model that's fast, accurate, and holds up at production scale. Tencent's Hunyuan team just open-sourced a framework built to do exactly that.

AngelSpec is a torch-native, end-to-end framework for training speculative decoding draft models, covering both the training pipeline and deployment integration. Built to serve Tencent's own Hunyuan 3 (Hy3) model in production, a 295B MoE with roughly 21B active parameters per forward pass, it shipped as a production system before it shipped as open source.

The numbers that matter

On Hy3-A21B, the top-performing architecture (DFly) delivers a 1.98 to 2.40x average speedup over the autoregressive baseline across all concurrency levels (4 to 64) and all six benchmarks, peaking at 2.86x on code and math tasks. It runs 10.5 to 11.8% faster than DFlash, the next best option.

The more revealing metric is mean accepted token length, which measures how many draft tokens the target model accepts before rejecting one. Fewer rejections mean fewer full forward passes, which translates directly to throughput. DFly reaches a mean accepted length of 4.79, about 30% above DFlash (3.69) and 60% above MTP (3.00), climbing to 5.52 on HumanEval.

Six architectures, one training pipeline

Rather than committing to a single drafting strategy, AngelSpec supports six distinct architectures, all trained through the same pipeline with a config change:

• DFly — Block-parallel drafting with hybrid target conditioning and a hidden-correction autoregressive head. The top performer across benchmarks.
• DFlash — A lightweight block diffusion model that generates draft tokens in a single forward pass, conditioned on hidden states extracted from the target model.
• DFlare — DFlash extended with learnable per-layer target fusion for stronger conditioning.
• Eagle3 — An autoregressive approach using test-time training (TTT) with input fusion, based on the EAGLE lineage.
• DSpark — A hybrid combining a DFlash backbone with an EAGLE-style autoregressive head.
• MTP — A full MoE decoder layer as the draft head, native to Hy3's architecture.

How the training system works

AngelSpec uses a disaggregated design: inference and training run on separate GPU pools. Inference engines run the frozen target model and extract multi-layer hidden states, which a Mooncake store streams over RDMA directly to the FSDP2 training workers. A controller handles batching, backpressure, and evaluation. The two pools scale independently, built on TorchSpec from LightSeek Foundation.

This matters because generating hidden states from the target model is typically the bottleneck. Decoupling the two sides lets you add inference GPUs without touching the training setup, and vice versa. Long-sequence training is supported via Ulysses sequence parallelism for 128k+ contexts.

The MTP training path uses test-time training (TTT), where the model trains through multi-step rollouts that mirror how it runs at inference. This on-policy approach closes the gap between training distribution and deployment, and AngelSpec does it with memory overhead close to a single causal forward pass.

What makes DFly faster

AngelSpec supports multiple composable loss functions: cross-entropy, top-k KL divergence, LK loss, and D-PACE weighting, mixable through configuration to tune draft quality against training stability. DFly's edge over DFlash comes from its hidden-correction head: after block-parallel drafting, an autoregressive correction step refines the output using the target model's hidden states, raising acceptance rates without giving up the parallelism of block generation.

What's available now

The release includes the AngelSpec training framework, Hy3-A21B MTP and DFly drafter weights, and training code. Pre-trained draft models on Hugging Face include:

• AngelSlim/Hy3-DFly-Block8 — DFly drafter for Hy3, no-think mode
• AngelSlim/Hy3-DFly-Block8-Think-High — DFly drafter for Hy3, high-think mode (for reasoning workloads)
• AngelSlim/Hy3-MTP-TTT3 — MTP drafter for Hy3
• AngelSlim/Qwen3-8B-DFly-Block8 — DFly drafter for Qwen3-8B
• AngelSlim/Qwen3-8B-MTP-TTT3 — MTP drafter for Qwen3-8B

Getting started takes two commands:

pip install -e ".[vllm]"
pip install mooncake-transfer-engine
# Single-node quickstart (8 GPUs: 4 inference + 4 training)
./examples/qwen3-8b-dfly/run.sh

The framework supports vLLM as the primary inference backend, with SGLang and HuggingFace also available. It's free and open-source under the AngelSpec License.

Where this lands in the broader field

Speculative decoding has moved fast as a research area. The field progressed from simple single-head approaches like Medusa to more sophisticated block-parallel methods like DFlash, accepted at ICML 2026. Most teams deploying speculative decoding today either use MTP heads baked into models like DeepSeek or Hunyuan, or rely on pre-trained EAGLE-style drafters from the community. Training a custom block-parallel drafter from scratch meant assembling a pipeline from parts.

AngelSpec packages the full stack: architecture options, disaggregated training infrastructure, acceptance-aligned objectives, and online evaluation, into a single repo with working examples for both Qwen3-8B and Hy3. For teams running large MoE models in production where the decode step is the primary bottleneck, that's a credible path to training high-quality custom draft models rather than waiting for the base model provider to ship one.