PrismML Shrinks Bonsai 27B to 3.9 GB so It Runs on an iPhone
PrismML's Bonsai 27B squeezes a full 27B reasoning model into 3.9 GB, making it the first model of its class to run on a phone.
- PrismML released Bonsai 27B, the first 27B-class model to run on a phone, at just 3.9 GB (1-bit) and 5.9 GB (ternary).
- The 1-bit variant retains 90% of full-precision performance; the ternary variant retains 95%, with math and coding nearly untouched.
- Built via Quantization-Aware Training on Qwen3.6 27B -- weights are constrained to {-1, +1} or {-1, 0, +1} end-to-end, no higher-precision layers.
- Supports 262K-token context, multimodal vision input, MCP tool use, and speculative decoding for a 1.37x lossless speedup.
- Available now on Hugging Face (GGUF, MLX, MLX Swift) and free via Together AI API, under Apache 2.0.
- Enables zero-marginal-cost agentic loops on-device, with private data never leaving the machine -- a new architecture for local AI agents.
PrismML just dropped Bonsai 27B, and the headline is hard to argue with: a 27-billion-parameter multimodal reasoning model that fits in 3.9 GB and runs on an iPhone. For context, the same model in standard 16-bit precision weighs 54 GB. Even a well-optimized 4-bit build clocks in at 18 GB. The Bonsai 27B 1-bit variant is smaller than most full-precision 2B models.
Two variants, one goal
The release ships two operating points, each targeting a different hardware tier:
- Ternary Bonsai 27B -- 5.9 GB, 1.71 effective bits per weight. Each weight is one of three values: {-1, 0, +1}, with a shared FP16 scale factor per group of 128 weights. Targets laptop-class deployment.
- 1-bit Bonsai 27B -- 3.9 GB, 1.125 effective bits per weight. Binary weights ({-1, +1} only), maximum compression. Targets phones.
Both variants are multimodal, with the vision tower shipping in a compact 4-bit form so on-device workflows can handle screenshots, documents, and camera input, not just text. Bonsai 27B carries a full 262K-token context and supports speculative decoding, compounding speed with lossless draft-and-verify acceleration. Everything is open-sourced under Apache 2.0.
How they got there
The key distinction from conventional quantization is that PrismML sidestepped the degradation problem by abandoning post-training quantization entirely -- Bonsai was built from the ground up as a native low-bit architecture. Standard quantization takes a trained full-precision model and rounds its weights down after the fact, which bleeds quality. PrismML instead uses Quantization-Aware Training (QAT), meaning the model learns with the ternary or binary constraint baked in from the start.
By enforcing ternary constraints in the forward pass and using full-precision gradients in the backward pass, the model learns feature representations optimized for ternary space, minimizing performance degradation even at extremely low bits. The full training algorithm is described as proprietary Caltech IP, but the paradigm aligns with established BitNet-style QAT research.
The architecture is derived from Qwen3.6 27B, a hybrid-attention causal language model with roughly 75% linear attention and 25% full attention, with the architecture itself left unchanged. The low-bit representation runs end-to-end across the language network -- embeddings, attention, MLPs, and the LM head -- with no higher-precision escape hatches. PrismML also trained custom DSpark speculative-decoding drafter layers on top, giving a lossless 1.37x decode speedup on the CUDA serving path.