DeepSeek V4.1 Flash shifts coding-agent serving economics

New AA-AgentPerf results compare how efficiently NVIDIA and AMD accelerators serve DeepSeek V4.1 Flash under concurrent coding-agent workloads. At a target of 100 output tokens per second per agent, NVIDIA’s GB300 NVL72 leads on throughput per megawatt, while an eight-GPU B300 node remains within roughly 20% across most of the benchmark curve.

A benchmark built around growing context

AA-AgentPerf replays multi-turn coding sessions whose context grows with each interaction. Earlier attention states remain in the key-value cache, commonly called the KV cache, so only new input tokens require prefill processing. Token generation is the decode phase. Tool calls introduce pauses between turns, and some trajectories exceed 100,000 input tokens.

The benchmark reports aggregate output throughput normalized by one megawatt of runtime power, alongside per-agent output speed at different concurrency levels. These views show how user density affects responsiveness. The power metric covers accelerator runtime consumption; capital cost, cooling, networking, and facility overhead require separate analysis.

GB300 leads; B300 stays close

At 100 output tokens per second per agent, the reported DeepSeek V4.1 Flash results rank the systems as follows:

DeepSeek V4.1 Flash can run on as few as two GPUs, reducing the advantage of rack-scale capacity for this model. The eight-GPU B300 configuration stays close to the 72-GPU GB300 system across most concurrency levels, which gives operators another option when rack density, deployment size, or acquisition cost constrains the design.

At batch size one, the Blackwell systems and AMD MI355X produce roughly 600 to 670 tokens per second for a single agent. NVIDIA’s Hopper-generation H100 and H200 reach about 372 to 392 tokens per second in the same benchmark.

Asymmetric compute cuts prefill cost

DeepSeek V4.1 Flash uses a native multimodal mixture-of-experts architecture with 552 billion backbone parameters and a context window of up to one million tokens. Its 40-layer causal encoder-decoder has 20 encoder layers and 20 decoder layers. The model activates about 8 billion parameters per token during prefill and 16 billion during decode.

That compute split suits coding agents because their accumulated prompts are often much longer than their responses. Processing fewer active parameters during prefill lowers the compute required to ingest source files, tool results, and earlier turns.

Cache compression raises session density

A KV cache stores the attention state for earlier tokens, allowing the model to continue a conversation without recomputing the full history. Long contexts consume substantial accelerator memory, so cache size can limit the number of simultaneous agents before raw compute capacity is exhausted.

The model combines several techniques to reduce that footprint:

• Compressed Sparse Attention 2 limits the attention state used for long contexts.
• FP4 KV storage keeps the main cache in four-bit precision.
• SWA Bounded Replay and Single-Pass mHC reduce retained state and repeated processing.
• Engram conditional memory retrieves learned token-pattern representations when needed.
• DSpark speculative decoding accelerates generation by proposing and verifying tokens in groups.

In the reported configuration, these mechanisms reduce the global KV cache to 890 bytes per token. The persistent cache is about one-eighth the size of DeepSeek V4 Flash’s. Lower per-token memory use allows more long-running agent sessions to share an accelerator.

Engram adds a separate 196-billion-parameter memory containing token-pattern embeddings. That memory can reside in CPU host memory, preserving accelerator capacity for active model weights and the KV cache.

Flash tops Pro on score and efficiency

V4.1 Flash has a 552-billion-parameter backbone and activates 16 billion parameters during decode, compared with 1.6 trillion backbone parameters and 49 billion active parameters for DeepSeek V4 Pro 0813. Flash scores 39 on the Artificial Analysis Intelligence Index, while Pro scores 36.

At 100 output tokens per second per agent, Flash delivers between 1.3 times the throughput per megawatt on GB300 and 6.2 times on B200 relative to V4 Pro 0813. The size of that range also shows why model comparisons need hardware-specific measurements rather than a single efficiency figure.

The updated leaderboard covers V4.1 Flash, V4 Pro 0813, Kimi K3, and GLM-5.2/5.3. Artificial Analysis says it plans to publish launch-day measurements for future open-weight models.

How to use the results

Developers sizing coding-agent infrastructure can apply the benchmark in several concrete ways:

• Compare deployment sizes. A model that fits on two GPUs may reduce the economic advantage of a rack-scale system.
• Model prefill separately. Input-heavy agent sessions benefit from architectures that activate less compute while ingesting context.
• Track cache bytes per token. KV compression directly affects how many long-context sessions fit in accelerator memory.
• Measure low-concurrency latency. The Hopper systems trail Blackwell and MI355X on single-agent speed in this run.
• Test complete configurations. Model architecture, inference software, quantization, interconnects, and accelerator choice all affect the result.

The benchmark supports evaluating the model, serving stack, and hardware as one deployment configuration. Teams should validate shortlisted systems with their own prompt lengths, tool-call cadence, response targets, concurrency patterns, and power assumptions before committing capacity.