Ternary Bonsai v2 Squeezes a 27B Model Into 7.66 GB
A ternary-quantized Qwen3.8-27B derivative packs uncensored reasoning, a one-row 'thinking fix,' and multi-token prediction into a 7.66GB file.
Takeaways
- Abliterated build of PrismML's Ternary Bonsai 2 27B, a Qwen3.8-27B derivative at 1.76 bits per weight.
- Scores 0.941 on StrongReject hard prompts, beating other abliterated Bonsai builds (0.67 to 0.78).
- One-row edit to the
</think>output logit fixes reasoning that never closes before token budget runs out. - MTP head delivers ~40% faster decode (69 to 97 tok/s on a 3090) with 0.658 draft acceptance.
- Only 0.95% of file bytes changed from stock; edits applied in-place to packed ternary weights, no re-quantization.
- Requires PrismML's llama.cpp fork; mainline llama.cpp, Ollama, and LM Studio cannot load the PQ2_0 format.
A community release called Ternary Bonsai v2 combines end-to-end ternary quantization, packed-weight refusal editing, and a multi-token prediction head. Its MTP variant occupies about 7.66 GB on disk. In the model card’s RTX 3090 test, speculative decoding increased throughput from 69.3 to 96.8 tokens per second. The release has passed 36,000 downloads on Hugging Face.
A 27B model in a 7.66 GB package
The release builds on PrismML’s base GGUF, a ternary-quantized version of Qwen3.8-27B. Each weight has one of three values, -1, 0, or +1. A shared scale for each group of 128 weights converts those values into the magnitudes used during inference.
PrismML reports an effective density of 1.76 bits per weight and a 5.9 GB footprint for the base language model. The model card advertises a 262K-token context window and reports 98.2% retention of the full-precision parent’s aggregate benchmark performance. The larger 7.66 GB MTP file includes the additional draft-head tensors used for speculative decoding.
Loading either file requires PrismML’s llama.cpp fork. Stock MLX and llama.cpp loaders do not apply the Hadamard activation transform required by this weight format.
Behavior edits stay inside packed weights
“Abliteration” refers to weight edits intended to weaken a model’s learned refusal behavior. The author reports modifying 99 of 851 tensors, targeting the ffn_down, ssm_out, and attn_output tensor families across blocks 15 through 63.
The edits operate directly on the packed ternary representation, avoiding a dequantization and re-quantization cycle. According to the model card, the changed data accounts for 0.95% of the file’s bytes.
A separate 1,250-byte edit adjusts one output-layer row associated with the