STDec Makes Diffusion LLMs 14x Faster Without Touching Model Weights
A training-free decoder exploits spatial and temporal stability in diffusion language models to deliver up to 14x throughput gains without hurting accuracy.
- STDec is a training-free decoder for diffusion LLMs that replaces global confidence thresholds with per-token adaptive ones. Paper
- It exploits two patterns: decoded tokens cluster near prior neighbors, and predicted IDs stabilize several steps before commit.
- Delivers up to 14.17x speedup on MBPP with LLaDA while matching baseline accuracy.
- Averages 10.03x speedup on Dream-7B and 7.60x on LLaDA-8B across code and math benchmarks.
- Works on multimodal dLLMs too, with 3.57x average throughput on LaViDa-Reason tasks.
- Stacks with existing cache-based accelerators like dKV-Cache and Fast-dLLM instead of replacing them. Project page
STDec replaces the global confidence cutoff used by diffusion large language model decoders with a threshold tailored to each token and denoising step. In the authors’ paper, the training-free method reaches a reported 14.17× speedup on MBPP with LLaDA while preserving a comparable benchmark score.
A diffusion large language model generates text by repeatedly replacing masked positions, allowing many tokens to resolve during the same pass. Each pass requires model computation, so committing more reliable tokens earlier can reduce latency, forward passes, and inference cost.
One cutoff ignores two signals
Conventional dLLM decoders score every masked position during each denoising step, then commit tokens that clear a global confidence threshold or rank among the top candidates. Positions below the cutoff remain masked until a later pass, even when their local context or prediction history indicates that the model has settled on an answer.
The authors identify two recurring patterns in their decoding analysis:
- Spatial consistency: More than 90% of decoded tokens appear near at least one previously decoded neighbor. Existing context around a masked position often makes its prediction safer to commit.
- Temporal consistency: Nearly 95% of decoded tokens preserve the same top-ranked token ID for at least one earlier step. The predicted ID can stabilize before its confidence clears a global threshold.
STDec lowers the bar selectively
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
