Qwen3.6 Runs Smarter on Mac by Giving Sensitive Layers More Bits
A 27B-parameter Qwen variant arrives on Apple Silicon with per-layer mixed-precision quantization, beating uniform 4-bit at the same memory budget.
PRO- Qwen3.6-27B-OptiQ-4bit ships as a 16.5 GB mixed-precision MLX quant for Apple Silicon.
- OptIQ assigns per-layer bit-widths via KL-divergence sensitivity: 247 layers at 8-bit, 343 at 4-bit.
- Average 4.5 bits-per-weight, same footprint as uniform quant but higher Capability Score.
- Loads directly into stock
mlx-lm, no custom runtime required. - Bundled Multi-Token Prediction head delivers ~1.4x decode speedup via
optiq serve --mtp. - Full mlx-optiq stack adds KV cache compression, LoRA, and hot-swap adapters.
A new 27-billion-parameter Qwen variant has landed on the MLX community hub, and it ships with a quantization trick that abandons the one-size-fits-all approach most local LLMs use today. Qwen3.6-27B-OptiQ-4bit is a mixed-precision conversion of Qwen3.6-27B that fits in roughly 16.5 GB of unified memory and is optimized specifically for Macs running MLX.
The headline is not the base model. It is the OptIQ method behind it, which decides bit-widths on a per-layer basis rather than crushing every weight matrix down to the same precision.
Not every layer deserves the same bits
Stock 4-bit quantization treats all layers equally. OptIQ rejects that premise. The toolkit observes that some layers are 50 times more sensitive to quantization than others, measures this once per model, and assigns bits per-layer at the same average bits-per-weight.
The mechanism is straightforward in principle. A per-layer KL-divergence sensitivity pass feeds a greedy knapsack solver, sensitive layers stay at higher precision, and the rest get aggressively quantized. For this specific checkpoint, that produced the following allocation:
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
