vLLM v0.26.0 is a dense release. This release features 411 commits from 212 contributors, including 61 first-timers. That contributor count is a signal worth paying attention to: vLLM is no longer just an NVIDIA project. AMD, Intel, IBM, and a growing list of model labs are all pushing code in. The result is a release that touches every layer of the stack, from low-level CUDA kernels to the Rust frontend and the OpenAI-compatible API.
The headline: a 1T-parameter model lands with Day-0 support
The most newsworthy addition is full support for the Inkling model family from Thinking Machines Lab. TML Inkling is a 1T-parameter multimodal model that natively accepts text, image, and audio inputs and generates text with up to 1M context length. It introduces several novel architecture components , relative attention, short convolution, and shared expert sinks , all of which are now efficiently integrated into vLLM.
The architecture is worth understanding before you try to deploy it. The model uses 66 layers, including 11 full-attention layers and 55 sliding-window attention layers. This heavy reliance on sliding-window attention is central to making million-token context practical. Instead of RoPE, Inkling uses relative attention, adding a learned relative-position term to attention logits. Each layer also applies a short convolution with window size 4 to attention keys, values, output, and MoE output.
On the serving side, Inkling includes eight MTP heads for speculative decoding , a technique where the model drafts multiple tokens in one forward pass and verifies them together , allowing up to nine tokens per step. vLLM reports up to 380 tokens per second per user with MTP on four NVIDIA GB200 GPUs, versus 140 tokens per second without MTP. vLLM also provides full feature parity, including LoRA, TP/DP/EP/PP parallelism, prefix caching, and disaggregated serving.
- Hardware target: Runs on both Hopper (H200 and H20) and Blackwell for BF16; NVFP4 variant targets NVIDIA Blackwell (B200/GB200).
- Scale: 975B total parameters, 41B active per forward pass, 1M-token context window.
- Quantization:
Don't miss what's next in AI
Join 300,000+ engineers and researchers who get the signal, not the noise.
- Full access to in-depth AI research breakdowns
- Be the first to know what's trending before it hits mainstream
- Daily curated papers, repos, and industry moves