Skip to main content
Back

Peking University's Saber Runs Diffusion Code Models 3.5x Faster Without Losing Quality

A training-free sampling algorithm for diffusion language models delivers 251% faster code generation while lifting Pass@1 accuracy by 1.9%.

  • Saber is a training-free sampler for diffusion language models that averages 251% faster inference with +1.9% Pass@1 on code tasks.
  • Adaptive Acceleration sets a moving confidence threshold from prior unmask decisions, unmasking few tokens early and many late.
  • Backtracking Remasking re-scores committed tokens each step and reverts those whose confidence dropped most under new context.
  • On HumanEval with LLaDA-8B, Pass@1 rises 43.3% to 45.1% while wall-clock drops from 2:11 to 41 minutes.
  • Works out-of-the-box on LLaDA, Dream, and DiffuCoder, and generalizes to MATH-500 and ARC-Challenge reasoning benchmarks.
  • Reference implementation is open source at github.com/zhaoyMa/Saber.

Diffusion language models were supposed to fix the slow, one-token-at-a-time bottleneck of autoregressive generation. They can fill in many masked positions per step and use bidirectional context, which suits structured outputs like code. In practice, cranking up the parallelism to actually harvest that speedup makes quality fall off a cliff. A new paper called Saber, from Peking University and Alibaba's Tongyi Lab, proposes a training-free sampling recipe that breaks the trade-off for code generation, lifting Pass@1 by 1.9% on average while running about 3.5x faster.

Why parallel decoding keeps collapsing

A diffusion language model (DLM) starts from a sequence of [MASK] tokens and iteratively unmasks positions across many denoising steps. Unmask one high-confidence token per step and you get quality similar to autoregressive models, but you also burn as many forward passes as the sequence is long. Unmask many tokens per step and you skip forward passes but risk committing to bad tokens.

The paper pins the collapse on two structural properties of DLM sampling:

  • Non-uniform difficulty. Early in decoding, the context is mostly mask tokens, so the model is uncertain. Later, once scaffolding like function signatures, brackets, and variable names is in place, predictions become much easier. A static parallelism setting is either too timid at the end or too reckless at the start.
  • Irreversible errors. Unlike autoregressive models, a DLM decides both which position to fill and what to put there. Once a token is unmasked in mainstream samplers, it is locked in forever. If an early guess conflicts with the context that emerges later, that error corrupts every subsequent prediction.