vLLM v0.26.0 Ships Day-0 Support for Inkling's 1T-Parameter Multimodal Model
vLLM v0.26.0 ships 411 commits with Day-0 Inkling support, per-group attention backends, tiered KV offloading, and sweeping DeepSeek-V4 speedups across all hardware
- vLLM v0.26.0 ships with 411 commits from 212 contributors, including Day-0 support for the 1T-parameter TML Inkling multimodal model.
- Inkling reaches up to 380 tok/s/user on 4x GB200 GPUs with MTP speculative decoding; supports text, image, and audio with 1M-token context.
- Attention backends are now selectable per KV-cache group, fixing memory waste in hybrid-architecture models like Gemma-3 and Inkling.
- Tiered KV offloading gains an object-store secondary tier (GPU HBM → CPU DRAM → object storage), with DP-replica-aware tiering and new observability metrics.
- DeepSeek-V4 gets ~4.7% cumulative TPOT improvement via a specialized routing kernel, fused_topk_bias (1.5–2x), and redundant copy removal across NVIDIA, ROCm, and XPU.
- Breaking changes: Transformers 5.13.0, FlashInfer 0.6.14, NIXL 1.3.1 required; TeleChat, Persimmon, and Fuyu models removed. Full release notes.
vLLM v0.26.0 is a dense release. This release features 411 commits from 212 contributors, including 61 first-timers. That contributor count is a signal worth paying attention to: vLLM is no longer just an NVIDIA project. AMD, Intel, IBM, and a growing list of model labs are all pushing code in. The result is a release that touches every layer of the stack, from low-level CUDA kernels to the Rust frontend and the OpenAI-compatible API.
The headline: a 1T-parameter model lands with Day-0 support
The most newsworthy addition is full support for the Inkling model family from Thinking Machines Lab. TML Inkling is a 1T-parameter multimodal model that natively accepts text, image, and audio inputs and generates text with up to 1M context length. It introduces several novel architecture components , relative attention, short convolution, and shared expert sinks , all of which are now efficiently integrated into vLLM.
The architecture is worth understanding before you try to deploy it. The model uses 66 layers, including 11 full-attention layers and 55 sliding-window attention layers. This heavy reliance on sliding-window attention is central to making million-token context practical. Instead of RoPE, Inkling uses relative attention, adding a learned relative-position term to attention logits. Each layer also applies a short convolution with window size 4 to attention keys, values, output, and MoE output.
On the serving side, Inkling includes eight MTP heads for speculative decoding , a technique where the model drafts multiple tokens in one forward pass and verifies them together , allowing up to nine tokens per step. vLLM reports up to 380 tokens per second per user with MTP on four NVIDIA GB200 GPUs, versus 140 tokens per second without MTP. vLLM also provides full feature parity, including LoRA, TP/DP/EP/PP parallelism, prefix caching, and disaggregated serving.
- Hardware target: Runs on both Hopper (H200 and H20) and Blackwell for BF16; NVFP4 variant targets NVIDIA Blackwell (B200/GB200).
- Scale: 975B total parameters, 41B active per forward pass, 1M-token context window.
- Quantization: