Solo Dev Shrinks Qwen3.6 to 27B and Bolts on 2-Billion-Row Memory
A solo researcher shipped a 27B Mixture-of-Experts model that runs on stock llama.cpp, grafting Qwen3.8-Flash-Next's n-gram memory into a pruned Qwen3.6 body.
Takeaways
- Whittle-Next-27B-A3B is a 27B-total, 3B-active MoE pruned from Qwen3.6-35B-A3B.
- Features a 2B-row hashed n-gram memory transferred verbatim from Qwen3.8-Flash-Next's table.
- Distilled from Qwen3.8-27B via logit distillation on 1,840 complete thinking traces.
- Scores 86.5% on GSM8K and runs on stock llama.cpp with no patches.
- Memory lives as a single lookup tensor, can be offloaded to system RAM for 16 GB cards.
- Research preview built by one developer on rented GPUs, Apache-2.0 licensed.
Whittle-Next model is a research preview that combines expert pruning, knowledge distillation, hyper-connections, and a 2-billion-row hashed n-gram memory. The one-person project was trained with rented GPU time and released with its training scripts, adapters, and logs under Apache-2.0.
The model contains 27 billion parameters while activating roughly 3 billion for each token. That distinction comes from its mixture-of-experts architecture, which routes each token through a small subset of specialized feed-forward networks. It uses the Qwen3.8-Flash-Next qwen4_exp format, a body derived from Qwen3.6-35B-A3B, and knowledge distilled from Qwen3.8-27B.
A 35B body, cut to 27B
The author reduced Qwen3.6-35B-A3B from 35 billion to 27 billion total parameters by removing experts layer by layer. The routing configuration still activates about 3 billion parameters per token, preserving the original model’s sparse-compute profile.
A 2-billion-row hashed n-gram table supplies an additional form of memory. N-grams are short token sequences, and hashing maps them to fixed table rows that the model can retrieve at every layer. This gives the network direct access to stored sequence features without placing the entire table in the transformer’s active computation.
The memory was transferred from Qwen3.8-Flash-Next after a least-squares mapping failed to reproduce the source table. That fit left 94% of the variance unexplained. The author instead retained eight of the source model’s 16 heads, selected the most frequently used 25% of rows, and copied those values directly into the student table’s dimensions.