Mia's AI Lab Squeezes DeepSeek V4.1 Flash's 552B Model onto Two Desktop Nodes
A community quantization squeezes DeepSeek's 552B multimodal MoE onto two desktop DGX Sparks, holding decode speeds across half a million tokens of context.
- Mia's AI Lab released a 2.9-bit EXL3 quantization of DeepSeek V4.1 Flash, MIT licensed.
- Base model is a 552B multimodal MoE with 8B/16B activated per token and a 1M context window.
- Quantized weights are ~197 GB across 39 shards, runnable on 2x DGX Sparks via vLLM tensor parallel.
- Native FP4 KV cache at ~890 B/token keeps a full 1M context under 1 GB memory.
- Engram memory tables are not quantized; shards 47 and 48 from the original repo are required.
- Turnkey 2x DGX Sparks launch kit automates the split download and vLLM boot.
Two DGX Sparks Can Serve a 552B DeepSeek MoE
Mia’s AI Lab has released an EXL3 checkpoint of DeepSeek V4.1 Flash that runs across two NVIDIA DGX Spark systems. Tensor parallelism splits the model’s weights and computation between the nodes, while 2.9-bit quantization reduces the decoder checkpoint to about 197 GB.
DeepSeek V4.1 Flash is a multimodal Mixture of Experts model with 552 billion total parameters and 384 routed experts. Its router activates about 8 billion parameters per token while processing a prompt and 16 billion while generating output. The architecture also combines a causal encoder-decoder, CSA2 compressed attention, Engram lookup memory, and an in-checkpoint DSpark draft model for speculative decoding.
Inside the 2.9-bit build
The checkpoint was converted with exllamav3 1.4.2. Its quantization recipe assigns lower precision to the decoder’s routed experts while retaining more precision for attention, memory, prediction heads, and embeddings.
| Component | Stored precision |
|---|---|
| Decoder average | 2.90 bits per weight with --hq |
| Routed experts | A mix of K=3 and K=2 EXL3 quantization, weighted toward K=3 |
| Multi-token prediction tensors | 4 bits per weight |
| Attention, shared, and Engram WKV tensors | Typically 4 to 6 bits per weight |
| Embeddings | BF16 |
| Quantized package |
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
