vLLM's AFD Plugin Cuts DeepSeek-V3.2 Response Time by 47%

Serving a Mixture-of-Experts (MoE) model like DeepSeek-V3.2 is really two very different jobs crammed into one system. The attention layers are memory-bandwidth-bound and tightly coupled to request scheduling and the KV cache. The expert FFN layers are compute-bound and dominated by token routing across many GPUs. Running both on the same hardware topology means you're always making the wrong tradeoff for at least one of them. The vLLM AFD Plugin is a new experimental plugin that solves this by physically separating the two into independent, scalable services.

The plugin is now available under the vllm-project/afd-plugin GitHub repository, contributed by teams from Ascend, vLLM, StepFun, Ant Group, and FastAFD. It targets vLLM 0.19.1 and requires Python 3.10-3.13. Critically, it does not require forking vLLM -- it integrates through vLLM's standard plugin entry point and --additional-config channel.

Why splitting attention from experts actually matters

Attention-FFN Disaggregation (AFD) is an emerging MoE serving architecture where attention nodes and expert/FFN nodes are physically separate groups of GPUs. The core motivation is that attention is memory-bandwidth-bound while expert MLPs are compute-bound, so a homogeneous deployment cannot optimally utilize hardware for both. This mismatch gets worse as you scale expert parallelism (EP) -- the more GPUs you throw at experts, the more you're wasting on attention, and vice versa.

Multiple production systems have already shipped this architecture: MegaScale-Infer (ByteDance) demonstrated disaggregated expert parallelism at scale, Step-3 (StepFun) used model-system co-design for cost-effective decoding, and xDeepServe (Huawei CloudMatrix384) deployed it as a model-as-a-service. What's new here is that AFD is now available as an open, pluggable extension to vLLM -- the most widely used open-source inference engine.