Redis Creator antirez Runs DeepSeek V4 Flash at 35 Tokens per Second
antirez uploads custom GGUF quantizations of DeepSeek V4 Flash to Hugging Face, purpose-built to run the 284B MoE model on MacBooks with 128 GB of RAM
- antirez (Redis creator) uploaded custom GGUF quantizations of DeepSeek V4 Flash to Hugging Face, purpose-built for his ds4 inference engine.
- The quants use asymmetric 2-bit compression: only routed MoE experts are crushed to 2-bit; attention, routing, and shared experts stay at Q8.
- The Q2 variant fits in ~76-87 GB of RAM, enabling the 284B model to run on 128 GB MacBooks at ~26 tokens/second.
- Files are not compatible with llama.cpp or other generic GGUF runners; they require the companion DwarfStar 4 (ds4) engine.
- The ds4 server exposes OpenAI and Anthropic-compatible endpoints, making it a drop-in local backend for Claude Code and opencode.
- The repo has 623,000+ downloads; the project is alpha quality and expected to take months to stabilize.
Salvatore Sanfilippo, better known as antirez and the creator of Redis, has uploaded a set of hand-crafted GGUF quantizations of DeepSeek V4 Flash to Hugging Face under antirez/deepseek-v4-gguf. These are not generic quants produced by an automated pipeline. They are bespoke weight files designed from the ground up to work with his companion inference engine, DwarfStar 4 (ds4), and to squeeze a 284-billion-parameter Mixture-of-Experts model onto Apple Silicon machines starting at 128 GB of RAM.
What is actually in the repo
The quants are specific for the DS4 inference engine. They may work with other inference engines, but the MTP model requires a specific loader. The recommendation is to use the Q2 variant on 128 GB Mac machines and Q4 on machines with 256 GB or more RAM, optionally pairing either with an MTP model for speculative decoding.
Two main quantization levels are available:
- Q2 (~87 GB): The Q2 quantized model takes about 76 GB of RAM. Targets MacBook Pro and MacBook Pro M3 Max class machines with 96 or 128 GB.
- Q4 (~165 GB): The Q4 variant requires machines with at least 256 GB and runs comfortably on Mac Studio M3 Ultra with 512 GB.
- MTP (optional): A draft model for speculative decoding, providing an experimental speed boost for greedy decoding.
The quantization design: asymmetric by intent
The key insight behind these GGUFs is not just how aggressively they compress the model, but
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
