Mia AI Lab's DFlash2 Drafter Boosts Qwen3.8 Decoding Speed by 33%
Mia-AiLab shipped a 5.0bpw EXL3 quant of the DFlash2 speculative drafter for Qwen3.8-27B, cutting draft weight from 3.85 GB to 1.4 GB and lifting decode throughput 33 percent on DGX Spark.
- Mia-AiLab released a 5.0bpw EXL3 quant of the DFlash2 drafter for Qwen3.8-27B
- Draft shrinks from 3.85 GB bf16 to 1.4 GB, a 63 percent size cut
- End-to-end decode throughput jumps 33 percent on DGX Spark at acceptance parity
- Hits 47.5 tok/s HumanEval-class decode at T=0.6 with 4.43 accepted tokens per step
- Requires the MiaAI-Lab exllamav3 fork; stock ExLlamaV3 does not implement DFlash2
- Roughly 15 percent faster than the built-in MTP head, trading context headroom for speed
A 5-bit DFlash2 drafter raises reported Qwen3.8 decode speed by 33%
Mia AI Lab has released a 5-bit DFlash2 checkpoint for speculative decoding with Qwen3.8-27B. The EXL3 quantization reduces the drafter’s weights from 3.85 GB in bf16 to 1.4 GB. In maintainer tests on an NVIDIA DGX Spark with GB10, end-to-end decoding rose from 35.7 to 47.5 tokens per second while draft acceptance remained comparable to bf16.
Speculative decoding uses a fast model to propose tokens before the larger target model verifies them in parallel and commits the accepted prefix. Throughput depends on proposal quality and drafting cost, so reducing the drafter’s memory traffic can accelerate generation without changing the target model.
Inside the seven-token draft
The released checkpoint is an EXL3 quantization of z-lab and inco.ai’s 1.93-billion-parameter DFlash2 model. It serves one purpose: proposing token blocks for Qwen/Qwen3.8-27B inside an ExLlamaV3-compatible speculative generation loop.
With a block size of eight, DFlash2 proposes up to seven speculative tokens in one pass. It retains high-scoring candidates at each position, then uses a lightweight selector to trace a coherent sequence through them. Two-tap dynamic convolutions help preserve proposal quality across later positions in the block.
A 33% gain on DGX Spark
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
