vLLM Matches DeepSeek's DeepInfer Within 6% on DeepSeek-V4.1-Flash

After DeepSeek open-sourced DeepInfer, the inference engine behind its production API, the vLLM team tested whether an external serving stack could reproduce its performance. On the same model and hardware, vLLM reported decode throughput within 6% of DeepInfer. On SemiAnalysis’s AgentX benchmark, it also produced an estimated $38.96 in gross API revenue per GPU-hour at DeepSeek’s off-peak rates.

Six percent from DeepInfer

The comparison uses DeepSeek-V4.1-Flash, a 552-billion-parameter mixture-of-experts model with a 131,000-token context window. DeepSeek’s decode configuration sets next-n token prediction to three, allowing the model to propose three upcoming tokens during each decoding step.

The engines share substantial technical ancestry. DeepInfer was built primarily on vLLM, with ideas from SGLang and TensorRT-LLM, while vLLM has integrated DeepSeek libraries such as MegaAttention, MegaMoE, Mega-mHC, Mega-Gate, and DeepSelect. Shared CUDA kernels leave scheduling, memory management, communication, and compute overlap as the main sources of performance differences.

The evidence applies to one model family, NVIDIA Blackwell hardware, and specific benchmark configurations. It provides no performance estimate for H100-class GPUs, AMD accelerators, other models, or production traffic with a different request mix.

Putting $38.96 in context

The $38.96 figure converts benchmarked token throughput into hypothetical gross API revenue using DeepSeek’s off-peak prices. It excludes hardware acquisition or rental, electricity, networking, engineering, idle capacity, and other operating costs, so it cannot establish profit or a direct cost advantage over DeepSeek’s hosted API.

SemiAnalysis’s AgentX workload replays synthetic request patterns designed to resemble multi-step agent activity. DeepSeek’s published figures reflect its production traffic mix, which may differ in prompt length, cache-hit rate, output length, concurrency, and latency requirements. Those differences prevent a direct revenue comparison between the two systems.

The calculation still gives infrastructure teams a useful capacity reference: vLLM can process enough V4.1-Flash traffic on a GB300 NVL72 system to support DeepSeek’s published off-peak prices under the benchmark’s assumptions. A deployment model can combine that throughput with local GPU costs and expected utilization to estimate gross margin.

A faster lane for agent loops

Agent workloads often alternate between model generation and external tool calls, making per-user decode speed a constraint on the time required to reach the next action. vLLM’s specialized V4.1-Flash decode path reached 826 tokens per second for one user across four B300 GPUs, a 26% improvement over its standard path on the tested GB300 configuration.

Coding agents, browser agents, and other sequential systems benefit when each generation step completes quickly, even when aggregate server throughput remains unchanged. Teams optimizing these workloads should measure time to first token, inter-token latency, time to the next tool call, and total task duration alongside tokens per second.

Where the speed comes from

• Wide expert parallelism and EPLB. Mixture-of-experts models route each token through a subset of model experts. Wide expert parallelism distributes those experts across more GPUs, while expert-parallel load balancing, or EPLB, keeps work from accumulating on a few devices. Dual-batch overlap hides part of the communication cost behind computation.
• DeepSeek’s CUDA kernels. vLLM incorporates specialized routines from MegaAttention, Mega-mHC, Mega-Gate, DeepSelect, and related DeepSeek libraries instead of relying solely on general-purpose implementations.
• Kernel fusion. Combining adjacent GPU operations reduces memory transfers and launch overhead. The implementation fuses mHC side-stream work and combines all-reduce communication with surrounding operations.
• A smaller KV-cache footprint. V4.1-Flash uses Compressed Sparse Attention 2, a four-bit floating-point KV cache, and cross-layer KV-cache sharing. These techniques reduce the stored attention state to 890 bytes per token, allowing larger batches or longer contexts in the same GPU memory.
• Separated prefill and decode pools. Prefill processes the prompt and is compute-intensive, while token-by-token decode is usually limited by memory bandwidth. vLLM assigns those phases to separate GPU pools and transfers state through NIXL, NVIDIA’s data-movement library.

The deployment decision

The benchmark indicates that DeepInfer has no clear decode-throughput advantage over vLLM for the tested V4.1-Flash configurations on Blackwell. Developers evaluating self-hosting can therefore focus on the operational variables that determine total cost and service quality:

• Utilization: sustained request volume is needed to recover the cost of GB200 or GB300 capacity.
• Traffic shape: prompt length, output length, concurrency, and cache-hit rate can move results far from the benchmark.
• Latency targets: aggregate throughput can hide slow individual requests, so tail latency and per-user decode speed need separate tests.
• Data control: self-hosting provides direct control over retention, network boundaries, batching, and request logs.
• Operational load: production serving requires monitoring, autoscaling, upgrades, failure recovery, and capacity planning.

The published recipes include verified serving commands for individual GPU configurations, including TP2 with FlashInfer MLA sparse kernels on Blackwell and DEP4 or DEP8 layouts for expert parallelism. For agent workloads with variable request lengths and frequent tool calls, the specialized low-latency decode path is the most relevant starting point, followed by tests using the application’s own prompts, concurrency, and service-level targets.

Developers comparing self-hosting with DeepSeek’s API should calculate cost per delivered output token under realistic utilization, then account for latency, privacy requirements, and operational staffing. The reported results make vLLM a credible serving option for V4.1-Flash, while the final economics remain specific to each workload and infrastructure contract.