Baidu's FastDeploy v2.5 Ships Qwen3-VL Support and Aggressive 4-Bit Quantization
PaddlePaddle releases FastDeploy v2.5, adding Qwen3-VL support, W4AFP8 quantization, and 170+ bug fixes for production LLM serving
Takeaways
- FastDeploy v2.5 released: PaddlePaddle's LLM/VLM serving toolkit ships Qwen3-VL support, new quantization, and 170+ bug fixes. Release notes
- Qwen3-VL and Qwen3-VL MoE: Both dense and MoE variants of Alibaba's vision-language model now supported, with CUDA Graph acceleration.
- W4AFP8 quantization: New mixed-precision format (4-bit weights, FP8 activations) for memory-efficient large MoE model deployment.
- Multimodal PD disaggregation: Prefill-Decode separation, previously text-only, now works for VLMs including RDMA and dynamic C8 IPC.
- RL training APIs: New async pause/resume/update_weights endpoints and Rollout Routing Replay for tighter RL pipeline integration.
- Blackwell GPU support: NVFP4 MoE and FusedMoE now supported on NVIDIA SM100 (Blackwell) architecture.
FastDeploy v2.5, Baidu's PaddlePaddle-based LLM and VLM inference toolkit, just shipped its most substantial release yet. The update lands Qwen3-VL deployment support, a new W4AFP8 quantization method, deeper reinforcement learning training integration, and over 170 bug fixes and performance improvements across the board.
What FastDeploy actually is
FastDeploy is an open-source, production-grade serving toolkit built on top of PaddlePaddle. Think of it as Baidu's answer to vLLM: it exposes an OpenAI-compatible API, handles batching and KV cache management, and is designed to run at scale across a wide range of hardware. Unlike vLLM, it places a heavy emphasis on Chinese-ecosystem hardware (Kunlunxin XPU, Hygon DCU, Iluvatar GPU, Enflame GCU, MetaX GPU) alongside NVIDIA GPUs and Intel Gaudi.
The toolkit has accumulated 3.7k GitHub stars and 751 forks, with the develop branch seeing active commits from 80+ contributors. It is free and Apache 2.0 licensed.
The headline additions
The two most visible additions in v2.5 are model support and quantization:
- Qwen3-VL and Qwen3-VL MoE: Both the dense and mixture-of-experts variants of Alibaba's Qwen3 vision-language model can now be deployed, including full CUDA Graph support for lower-latency decode.
- W4AFP8 quantization: A new mixed-precision scheme that stores weights at 4-bit integer precision and activations at FP8 (8-bit floating point). This is more aggressive than W8A8 and targets memory-constrained deployments of large MoE models. It supports tensor parallelism greater than 1 and works with both the v0 and v1 model loaders.