Intelligent Internet's Meta-Zenith Agent Rewrote vLLM Kernels for a 4x Speedup

Intelligent Internet just published a case study that reads more like a research prompt than a product launch: point an autonomous agent at a single vLLM inference stack, give it one GPU, and let it optimize until it stops finding wins. The agent, called Meta-Zenith, ran 111 self-directed trials with no human in the loop and squeezed a 63.5% geometric-mean throughput gain out of stock vLLM v0.27.1 serving Qwen3.8-27B-NVFP4 on an RTX 5090.

The headline number is the long-context result. At 0K context the optimized build hits 185.5 tok/s versus 99.3 for stock (a 1.87x gain), at 8K it goes from 67.7 to 153.1 (2.26x), at 32K from 43.9 to 131.6 (3.00x), and at 65K from 29.0 to 119.0 tok/s, a 4.10x speedup. Stock vLLM falls off a cliff as context grows; the tuned fork barely slows down.

The agent that rewrote the kernels

Meta-Zenith is the second layer of Intelligent Internet's Zenith harness, an open-source system for long-running engineering agents. Zenith keeps an agent working, testing, and improving over the long haul; Meta-Zenith generates the right Zenith configuration for a new task. Here the task was kernel-level inference optimization inside a vendored vLLM tree, with bitwise-exact outputs as a hard constraint.

The loop was strict about what counted as a win. Every trial got profiled, checked by bitwise falsifiers so outputs stayed exact, and quality-gated against MMLU and a tool-use benchmark. Of the 111 trials, 25 were promoted into the release, 30 were held for later, 10 that were fast but lossy got auto-rejected, and 46 were screened out in minutes.

Where the speed actually came from

The optimization delta touches 24 upstream source files with roughly 3,300 lines added. The agent grouped its wins into a handful of areas:

• TurboQuant 4-bit KV decode: smaller decode tiles to reduce per-launch working data, context-dependent kernel selection, and CUDA Graph capture boundaries aligned with kernel dispatch.
• Native CUDA kernels: a head-parallel kernel for context regimes where independent head work is preferable, and a shared high-batch kernel that reuses dequantized KV data across compatible requests and heads.
• Fused GDN prefill: convolution outputs routed directly into the Gated Delta Network prefill computation, with Q/K normalization performed inside the fused path to avoid round trips through temporary tensors.
• Adaptive scheduling: hybrid KV-cache capacity reserved for batched prefills, FlashInfer tuning run before KV-cache allocation, and long prefill chunks adapted to the active requests' fair share of the token budget.
• Speculative decoding cleanup: the speculative GDN update reads Q, K, and V directly from the packed QKV representation instead of materializing intermediate tensors.

What you actually get

The release is a hardware-specific fork rather than a drop-in vLLM upgrade. The target is RadixArk/Qwen3.8-27B-NVFP4 on one RTX 5090 with sm_120, NVFP4 weights with TurboQuant 4-bit KV cache, the MTP speculative decoding path, and long-context inference up to 262,144 tokens. Shipped artifacts include the optimized vLLM source, a prebuilt wheel targeting CUDA 13.0 and Python 3.12, reproducible benchmark scripts, and raw logs. Everything is Apache-2.0.

Installation uses uv and a pinned requirements file. The prebuilt wheel is the recommended path because compiling the vendored CUDA extensions takes a while; a source build needs the CUDA 13.0 toolkit and TORCH_CUDA_ARCH_LIST=12.0.

Why this matters beyond one GPU

The interesting claim here goes beyond a 5090 serving a 27B model at 119 tok/s at 65K context, impressive as that is. An agent found the optimizations on its own, gated by hard correctness and quality checks, over a vendored codebase most engineers would consider too gnarly to hand-tune without a specialist team. The model weights are frozen and the agent does not rewrite its own code; what changes during a run is the shape of work, which workers exist, what testing layers are active, what skills have been learned, and how the plan is structured. The Intelligent Internet team calls this adaptive self-improvement rather than recursive self-improvement.

Caveats matter here. Throughput is client-observed and includes prefill time. The build has only been validated on one GPU with one model. Concurrency-2 and concurrency-4 gains are much smaller than the single-stream numbers, mostly in the 30-40% range once context grows past 16K. A good chunk of the gain also reflects headroom that a well-optimized upstream vLLM release would eventually close on its own.

Even with those qualifications, if you have a long-horizon engineering problem with a hard, measurable objective, a bitwise correctness oracle, and a benchmark you trust, this release is a concrete demonstration that agent harnesses can now do the work that used to require a performance engineer with a month of runway.