Prism ML Squeezes a 2B Vision Model Into 0.52 GB
Prism ML's 2B vision-language model squeezes into roughly half a gigabyte using ternary weights while keeping a 262K context window and image input.
- Prism ML released Ternary-Bonsai-2-2B-gguf, a 2B vision-language model with ternary weights.
- Language model fits in ~0.52 GB, down from ~3.8 GB at FP16, via PTQ1_0, PQ2_0 or Q2_0 packings.
- Derived from Qwen3.5-2B with hybrid 3:1 linear/full attention and a 262K-token context window.
- Vision comes through an optional 0.36 GB Q8_0 mmproj file, loaded only when images arrive.
- Requires the PrismML llama.cpp fork; only the Q2_0 file runs on upstream llama.cpp b11556+.
- Apache 2.0 licensed, targets on-device agents on phones, laptops, smart glasses and WebGPU.
Prism ML has released Ternary-Bonsai-2-2B as GGUF downloads for llama.cpp. The model compresses most transformer weights to three values and reduces the language-model file from roughly 3.8 GB at FP16 to 0.52 GB in the default packing. Compatible builds can run it through CUDA, Metal, or CPU backends, making the release relevant to developers building local multimodal tools under tight storage and memory limits.
Three values replace FP16 weights
Each quantized transformer weight becomes -1, 0, or +1. This ternary representation is often called 1.58-bit quantization because three possible states contain about 1.58 bits of information. Actual files require additional metadata, so their effective storage cost is higher.
The quantization covers embeddings, attention projections, MLP projections, and the tied language-model head. Each group stores an FP16 scale that restores an approximate magnitude during matrix multiplication. Normalization weights and the linear-attention state path remain in F32 to reduce numerical instability.
A blockwise Hadamard rotation changes the basis in which the weights are stored. The runtime applies the corresponding transform to activations during inference, which explains why these files require explicit support in llama.cpp rather than a generic low-bit loader.
Choose the packing for the runtime
The three downloads contain the same ternary weights but use different storage and unpacking layouts. GGUF packages model weights and metadata in the format consumed by llama.cpp.
| Packing | Size | Effective density | Runtime support | Best fit |
|---|---|---|---|---|
PTQ1_0 |
0.43 GB | About 1.83 bits per weight | Prism ML fork | Lowest storage and weight-memory use |
PQ2_0 |
0.52 GB |
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
