Skip to main content
Back

Qwen3.6 Shrinks to 3.4 GB After Cerebras REAP Deletes 90% of Experts

A community experiment uses REAP expert pruning to shrink Qwen3.6-35B down to 6B params and 3.4 GB, trading coherence for extreme portability

  • A community release applies REAP expert pruning at 90% to Qwen3.6-35B-A3B, shrinking it from 35B to 6.15B parameters and fitting it in a 3.4 GB GGUF file.
  • REAP (Router-weighted Expert Activation Pruning) is a one-shot method from Cerebras that removes low-impact experts without retraining, outperforming expert merging on generative tasks.
  • Two files are available: Q4_K_M at 3.4 GB (mostly coherent) and IQ1_S at 1.6 GB (largely incoherent).
  • The model requires a llama.cpp build with QWEN35MOE support; it runs via llama-cli, llama-server, Ollama, or llama-cpp-python.
  • The author labels it a meme and explicitly warns against using it in production -- it is a compression demonstration, not a deployable model.
  • The original REAP paper tested up to 50% pruning with near-lossless results; 90% is well past the quality cliff.

Someone just published a "35B" model that fits in a 3.4 GB file. The catch is baked right into the model card: this model is meant to be a meme. But underneath the self-deprecating honesty is a genuinely interesting experiment in how far you can push MoE compression before a model stops making sense.

The base model is already unusually efficient

Qwen3.6-35B-A3B is Alibaba's latest MoE model. The "A3B" in the name tells you the key number: the model has 35B total parameters across all expert modules, but only 3B are activated per token. That's the MoE trick -- you store a large pool of specialized sub-networks ("experts") and route each token to just a few of them at inference time.

The architecture is unusually dense with experts. Its MoE layer contains 256 experts in total, with 8 routed experts and 1 shared expert activated per token. That 256-expert pool is exactly what makes this compression experiment possible -- and extreme.

The technique: REAP, and what it actually does

The compression method used here is called REAP (Router-weighted Expert Activation Pruning), developed by Cerebras Research. The core argument is that pruning experts is superior to merging them for one-shot compression of MoE LLMs, achieving nearly lossless performance on generative tasks by minimizing the upper bound of the reconstruction error.

The key insight is about what goes wrong when you merge experts instead of removing them. For generative tasks like code generation, pruning low-impact experts is fundamentally better than merging them. Merging forces the router to use a static average of two experts for all inputs, destroying the model's ability to dynamically mix expert outputs depending on context -- what the Cerebras team calls "functional subspace collapse."