---
title: "Google DeepMind's DiffusionGemma Generates Text 4x Faster by Ditching Tokens"
subtitle: "Google DeepMind's DiffusionGemma generates 256 tokens in parallel, hitting 1,000+ tokens/sec on H100 — but trades raw quality for speed"
url: "https://alphasignal.ai/news/google-deepmind-s-diffusiongemma-generates-text-4x-faster-by-ditching-tokens"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "Google DeepMind"
company_url: "https://alphasignal.ai/company/googledeepmind"
published: "2026-06-10T16:06:44.000Z"
updated: "2026-06-16T19:51:15.589Z"
type: "News"
topics:
  - "Llms"
  - "Open Source"
subtopics:
  - "Small Models"
upvotes: 2479
read_time: "7 min"
source_url: "https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/?utm_source=x&utm_medium=social&utm_campaign=&utm_content="
comments: 0
---

# Google DeepMind's DiffusionGemma Generates Text 4x Faster by Ditching Tokens

> Google DeepMind's DiffusionGemma generates 256 tokens in parallel, hitting 1,000+ tokens/sec on H100 — but trades raw quality for speed

Google DeepMind just shipped something that looks less like a language model and more like an image generator applied to text. [DiffusionGemma](https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/) is a new experimental open-weights model that abandons the token-by-token generation loop entirely, instead denoising entire 256-token blocks in parallel , reaching over 1,000 tokens per second on a single NVIDIA H100 and up to 4x faster output than standard autoregressive models on dedicated GPUs.

This is not a quantization trick or a speculative decoding hack. It is a fundamentally different generation paradigm, and it is now available for free under an Apache 2.0 license on Hugging Face, Kaggle, and Vertex AI.

## The problem with the typewriter

Every major language model you have used , GPT, Claude, Llama, Gemma , works the same way under the hood: it predicts one token at a time, left to right, each step waiting for the previous one to finish.
Autoregressive models use causal masking so the model can only attend to previous tokens, which is what enforces the left-to-right generation constraint.
This works well in cloud deployments where servers batch thousands of user requests together to keep GPUs busy. But locally, for a single user,
the primary bottleneck is memory bandwidth , the model must repeatedly load weights from memory to generate text one token at a time.
Your GPU's tensor cores sit largely idle, waiting for the next "keystroke."

DiffusionGemma flips this.
By shifting the decode bottleneck from memory-bandwidth to raw compute, DiffusionGemma generates up to 4x faster token output, achieving over 1,000 tokens per second on a single NVIDIA H100 GPU.
On an H200,
the FP8 version reaches 1,008 tokens per second at batch size 1 on a single H100, and 1,288 on H200 , roughly six times a standard autoregressive baseline, according to vLLM benchmark results.

## How text diffusion actually works

The mechanism borrows directly from image generation. Think of how Stable Diffusion starts with pure visual noise and progressively refines it into a coherent image through many denoising passes. DiffusionGemma does the same thing, but over discrete text tokens:

1. **The canvas:** The model starts with a 256-token block filled with random placeholder tokens.
2. **Iterative denoising:** Over multiple forward passes, highly confident tokens get locked in and used as context clues to resolve adjacent positions , the whole sequence snaps into focus progressively.
3. **Self-correction:** If a token's confidence drops during a pass, the sampler can re-noise and replace it. This is something autoregressive models fundamentally cannot do once a token is committed.
4. **Long sequences:** For outputs longer than 256 tokens, each finalized block gets committed to a KV cache and the model starts a fresh canvas conditioned on the prior history , combining parallel block speed with sequential stability.

![DiffusionGemma before and after fine-tuning on Sudoku: the fine-tuned model correctly fills in the 9x9 grid](https://storage.googleapis.com/gweb-uniblog-publish-prod/original_images/sudoku_before_after11.gif)

The key architectural unlock is **bidirectional attention**.
Generating 256 tokens in parallel with each forward pass allows every token to attend to all others, which provides significant advantages for non-linear domains such as in-line editing and code infilling.
Standard autoregressive models can only look backward , they are blind to what comes next. DiffusionGemma sees the whole canvas at once.

## What it is built on

The model is built on the Gemma 4 architecture as a 26B-class Mixture-of-Experts model (25.2B total parameters), but only about 3.8B parameters are active per step , which is why Google labels it 26B A4B.
The MoE design keeps the active compute footprint small.
It fits comfortably within the 24GB VRAM limits of a consumer NVIDIA RTX 5090 or 4090 when quantized.
The diffusion mechanism itself is built on top of the Gemma 4 backbone, with a novel diffusion head added to handle the denoising loop , meaning
DiffusionGemma integrates cutting-edge Gemini Diffusion research with the intelligence-per-parameter of the Gemma 4 family.

It accepts text, image, and video inputs and produces text output
, making it a multimodal model despite its speed-focused design.

## The industry context: dLLMs have been brewing for years

This is not the first diffusion language model, but it is the most prominent open-weights one from a major lab.
Early efforts like D3PM, MDLM, and MD4 focused on exploring training objectives, noise scheduling, and parameterization methods.
More recent work includes LLaDA, the first large-scale diffusion LLM, and DiffuGPT/DiffuLLaMA, which adapt pre-trained autoregressive models to the diffusion framework.
On the commercial side,
higher-performance commercial diffusion LLMs such as Mercury (Inception Labs) and Gemini Diffusion were announced
before DiffusionGemma.
A year ago at Google I/O 2025, DeepMind showed a demo of a model that filled an entire screen with text at once. That technology was called Gemini Diffusion, it was closed, and reserved for a handful of selected testers.
DiffusionGemma is the open version of that bet.

DiffusionGemma is the first diffusion language model natively supported in the open-source vLLM inference platform
, which is a significant milestone for the ecosystem , it means the tooling most teams already use for serving can handle this new paradigm out of the box.

## Where it shines, where it struggles

Google is unusually candid about the trade-offs.
Because it prioritizes speed and parallel layout generation, DiffusionGemma's overall output quality is lower than standard Gemma 4. For applications that demand maximum quality, Google recommends deploying standard Gemma 4.

Where DiffusionGemma has a *structural* advantage over autoregressive models:

- **Code infilling and in-line editing:** Bidirectional attention lets it fill gaps in existing code or text without the left-to-right blindness of standard models.
- **Complex markdown and structured output:** It can close formatting tags correctly because it sees the whole block simultaneously.
- **Constrained generation tasks:**
  On structured constrained tasks, including code infilling, template generation, and problems requiring bidirectional constraint propagation, the architecture has a structural advantage that fine-tuning can surface , as the Sudoku result demonstrates.
- **Low-latency local inference:** The speed advantage is strongest at low-to-medium batch sizes on a single accelerator.

Where it falls short:

- For short-form generation and summarization, quality is competitive. For complex reasoning, long-form writing, and code generation, autoregressive models win by a meaningful margin.
- The speed advantage collapses on low-compute GPUs (3060, 4060, laptops) and Apple Silicon.
  The gains require compute-bound hardware, not memory-bandwidth-bound unified memory architectures.
- There are no established LoRA/QLoRA recipes yet, and no community adapter ecosystem. Fine-tuning a diffusion model is conceptually different from autoregressive , you are tuning denoising behavior, not next-token prediction.

## How to run it today

The weights are released under the Apache 2.0 license on Hugging Face, Kaggle, and Vertex AI, so you can download, modify, and deploy the model commercially at no cost. You pay only for your own compute.
The model identifier on Hugging Face is `google/diffusiongemma-26B-A4B-it`.

Serving it with vLLM is already supported. Here is the launch command from the [official developer guide](https://developers.googleblog.com/en/diffusiongemma-the-developer-guide):

```
vllm serve google/diffusiongemma-26B-A4B-it \
  --max-model-len 262144 \
  --max-num-seqs 4 \
  --gpu-memory-utilization 0.85 \
  --attention-backend TRITON_ATTN \
  --generation-config vllm \
  --hf-overrides '{"diffusion_sampler": "entropy_bound", "diffusion_entropy_bound": 0.1}' \
  --diffusion-config '{"canvas_length": 256}' \
  --enable-chunked-prefill
```

Note that the generation pipeline requires custom sampling code for the masked diffusion loop ,
the model is hosted on the Google DeepMind Hugging Face organization, and you can load it using the standard transformers library, though the generation process requires custom sampling code for the masked diffusion pipeline rather than the standard `.generate()` method.
Fine-tuning support is available via Unsloth and NVIDIA NeMo, and [Hackable Diffusion](https://github.com/google/hackable_diffusion), a modular JAX toolbox, is provided for research-level experimentation.
Support for llama.cpp is coming soon.

## What this means for the field

DiffusionGemma is a signal that the industry is seriously exploring alternatives to autoregressive generation. If text diffusion can reach quality parity with autoregressive models , which many researchers believe is achievable , it would fundamentally change how we deploy LLMs.
The key assumption that needs updating: generation speed on local hardware is a hardware problem. DiffusionGemma reframes it as an architectural one. The bottleneck was never just the GPU , it was the sequential contract baked into every transformer decoder.

There is no technical reason why a diffusion model could not perform just as well as a more traditional large language model
, the focus here is on speed, and quality is a matter of further training and scaling. With Apache 2.0 weights, first-class vLLM support, and NVIDIA hardware optimization already in place, the community now has a serious open platform to close that gap.

## Sources

- [X post by @googledeepmind](https://x.com/googledeepmind/status/2064741061352636762)
- [DiffusionGemma: 4x faster text generation — Google Blog](https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/)
- [DiffusionGemma: The Developer Guide — Google Developers Blog](https://developers.googleblog.com/en/diffusiongemma-the-developer-guide)
- [Google's DiffusionGemma generates 256 tokens in parallel — VentureBeat](https://venturebeat.com/technology/googles-diffusiongemma-generates-256-tokens-in-parallel-and-self-corrects-as-it-goes)
- [NVIDIA Accelerates DiffusionGemma for Local AI — NVIDIA Blog](https://blogs.nvidia.com/blog/rtx-ai-garage-local-gemma-diffusion/)
- [DiffusionGemma — Google DeepMind](https://deepmind.google/models/gemma/diffusiongemma/)