Inception Labs' Mercury 2 Hits 1,000 Tokens per Second on Azure

Mercury 2, the diffusion-based language model from Inception Labs, is now live on Azure AI Foundry. The pitch is simple but striking: reasoning-quality output at over 1,000 tokens per second, at less than half the cost of comparable models. That's not a minor speed bump. It's a different architecture entirely.

Why the typewriter metaphor finally breaks

Every major language model you've used, GPT, Claude, Gemini, Llama, generates text the same way: one token at a time, left to right, each token depending on all the ones before it. It works, but it creates a hard ceiling on speed. Mercury 2 generates all tokens simultaneously and refines them through iterative denoising passes, a process borrowed from image generation models like Stable Diffusion.

The core insight is that text diffusion can work with discrete tokens using masking as the "noise" process, sometimes called discrete diffusion or absorbing diffusion, which sidesteps issues with continuous diffusion applied to language. In practice: rather than adding Gaussian noise to continuous embeddings, Mercury 2 uses a masking-based corruption process designed specifically for discrete tokens, producing more stable training and sharper convergence during inference.

Think of it less like a typewriter and more like an editor. The model starts with a canvas of masked tokens, then refines the whole draft in parallel, converging over a small number of steps. The number of denoising steps adjusts dynamically based on output complexity.

The numbers that matter

Mercury 2's standout claim is raw throughput: roughly 1,000 tokens per second versus about 89 tokens/sec for Anthropic's Claude Haiku 4.5 Reasoning and 71 tokens/sec for OpenAI's GPT-5 Mini. That's not 5x faster than slow frontier models. It's 5x faster than models already optimized for speed.