DeepSeek's DSpark Makes DeepSeek-V4 Generation 85% Faster at Any Scale

Speculative decoding has always had a dirty secret: it works brilliantly at low concurrency, and quietly falls apart as your server fills up. SGLang now integrates DSpark, a new speculative decoding framework from DeepSeek, that attacks this problem head-on with confidence-driven, variable-length verification. The result is the best throughput/latency tradeoff across the full concurrency range, from a single user to 256 concurrent requests.

Why standard speculative decoding breaks under load

The standard speculative decoding loop works like this: a small, cheap draft model proposes a block of K tokens, and the large target model verifies all of them in one forward pass. At batch size B, that means verifying B * K tokens every step. Parallel drafters efficiently propose long token sequences in a single forward pass, but suffer from rapid acceptance decay due to a lack of inter-token dependencies. Indiscriminately verifying these extended blocks wastes critical batch capacity on tokens with high rejection risks, severely degrading throughput in high-concurrency serving systems.

Speculation spends spare compute to skip memory reads, so its benefit collapses as concurrency rises and the GPU becomes compute-bound. At batch size one you have abundant idle compute and speculation is a clear win. On a server saturated with concurrent requests, the extra verification work competes for compute that is already busy, and the speedup trends toward zero or negative. Most frameworks just pick a fixed verification budget and hope for the best.

What DSpark actually does differently

The full name is Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation. The core idea is simple: draft more than one token, verify intelligently, and avoid spending expensive target-model compute on tokens that are unlikely to survive. DSpark achieves this through two interlocking mechanisms:

• Semi-autoregressive block drafter: DSpark utilizes a semi-autoregressive architecture, coupling a parallel backbone with a lightweight sequential module, to introduce intra-block dependency modeling and mitigate suffix decay. The sequential head (a Markov chain or small RNN) conditions each draft token on the previous one, keeping acceptance rates high across the whole block without the cost of full autoregression.