vLLM's DSpark Beats Every Fixed-Length Config on DeepSeek-V4 at Any Load
vLLM's new adaptive verification for DSpark automatically tunes speculative decoding depth per step, holding the Pareto frontier from concurrency 1 to 256 with a single config.
- Adaptive verification is live in vLLM main behind
enable_adaptive_verification, merged in PR #47808. - One config now covers all concurrencies: a single
num_speculative_tokens: 7setting holds the Pareto frontier from concurrency 1 to 256 on 8×B300 GPUs. - Per-step budget scheduling: DSpark's confidence head scores each draft token; vLLM allocates verification slots to the highest-survival tokens across the whole batch.
- Acceptance decay is steep: on DeepSeek-V4-Pro-0813, token 1 of a 7-token draft survives 70%+ of the time; token 7 survives less than 10%.
- Current limits: requires SM100 hardware (B300), no LoRA, no pipeline parallelism, no output logprobs, no eager mode.
- Free and open source as part of vLLM; no separate draft model download needed for DSpark on DeepSeek-V4.
Speculative decoding is one of the most powerful tricks in the LLM serving playbook: a cheap draft model proposes several tokens at once, and the full target model verifies them all in a single forward pass. The target model accepts the longest prefix consistent with its own distribution and appends one bonus token, accelerating generation without any quality loss. The catch has always been tuning the draft length. Too short, and you leave speed on the table at low load. Too long, and at high concurrency you're burning compute verifying tokens that will almost certainly be rejected.
DSpark, DeepSeek's speculative decoding framework, introduced a confidence head that scores each drafted token's probability of surviving verification. While parallel drafters efficiently propose long token sequences in a single forward pass, they suffer from rapid acceptance decay due to a lack of inter-token dependencies, and indiscriminately verifying these extended blocks wastes critical batch capacity on tokens with high rejection risks. vLLM now closes that gap with adaptive verification: instead of committing to a fixed draft length at deploy time, the engine decides per step how many tokens are worth verifying, based on live confidence scores.
Why the old approach breaks at scale
The acceptance rate of draft tokens is not flat across positions. On DeepSeek-V4-Pro-0813, the first token of a 7-token draft survives verification more than 70% of the time. The last one, less than 10%. Under the old fixed-length scheme, you had to pick one number for your whole deployment and live with it. No static num_speculative_tokens is optimal across concurrencies: the crossover moves with load and workload-dependent acceptance rates.
A uniform K, however well chosen, pays for this within-batch spread either in wasted verification (K too long for cold requests) or lost acceptance (K too short for hot ones). In practice, teams running DeepSeek-V4 at scale had to benchmark their typical traffic shape and pick a static number, then re-tune whenever load patterns changed.