PrismML compresses Qwen3.8 27B into 5.9 GB

PrismML has released Ternary Bonsai 2 27B, a quantized version of Qwen3.8 27B that stores its weights in 5.9 GB. The company reports an overall benchmark score of 83.9, compared with 85.4 for the full-precision model, yielding 98.2% performance retention with a weight package more than nine times smaller.

The Apache 2.0 weights are available on Hugging Face. PrismML also provides a browser-based WebGPU demo and custom runtimes for NVIDIA and Apple hardware.

A 27B model in a 5.9 GB package

Three values replace 16-bit weights

Ternary quantization maps each model weight to one of three values: -1, 0, or +1. Each group of weights shares an FP16 scaling factor, allowing the model to approximate a wider numerical range while keeping the individual weight codes compact.

At 1.76 effective bits per weight, 27 billion parameters require roughly 5.94 billion bytes in decimal units, which aligns with PrismML’s reported footprint. Every language-model layer uses the low-bit representation.

Actual inference memory exceeds 5.9 GB because activations, the key-value cache, temporary buffers, runtime code, and input data require additional space. The 262K-token context limit also describes the model architecture; practical context length depends on available memory and the runtime’s cache format.

Most benchmark loss lands in vision

Vendor-reported results. Delta is Bonsai 2 minus full precision.

Capability	Bonsai 2 27B	Qwen3.8 27B	Delta
Knowledge and reasoning	83.95	86.66	-2.71
Math	96.57	97.06	-0.49
Coding	81.58	82.17	-0.59
Agentic and tool calling	77.57	79.74	-2.17
Instruction following	82.66	81.25	+1.41
Vision	78.59	81.64	-3.05
Overall	83.9	85.4	-1.5

The smallest reported gaps appear in math and coding, at 0.49 and 0.59 points. Vision shows the largest decline, followed by knowledge and reasoning, then agentic tool use. Applications that depend on screen interpretation, long action sequences, or complex tool graphs need task-specific evaluation.

The instruction-following score rises by 1.41 points, a difference that may reflect evaluation variance rather than an improvement caused by quantization. Aggregate retention also hides variation among individual tasks and prompts.

All benchmark figures come from PrismML. The company’s white paper contains the per-benchmark results and compression methodology, while independent reproduction remains necessary for comparisons across runtimes and production workloads.

Generation two closes more of the gap

The original Bonsai 27B arrived two months earlier with reported retention of about 95%. Bonsai 2 raises that figure above 98% and targets the long agent trajectories, tool-calling loops, and multimodal tasks that exposed larger losses in the first release.

The base model also changed to Qwen3.8 27B, so the three-point retention gain combines a stronger foundation with changes to PrismML’s quantization and runtime stack. The release attributes the new model’s gains to higher benchmark retention, faster execution, and improved long-horizon agentic behavior.

Custom kernels carry the speed claim

PrismML reports peak generation rates of 143 tokens per second on an NVIDIA GeForce RTX 5090 and 44 tokens per second on an Apple M5 Max. These are maximum reported figures for specific systems, rather than expected rates across all prompt lengths, context sizes, and sampling settings.

The company also reports energy use of 0.581 mWh per generated token on an RTX 4090. PrismML describes that result as 40% more energy-efficient than an 8B full-precision model, although the comparison depends on the selected model, batch size, sequence length, power limits, and measurement method.

PrismML supplies CUDA kernels for NVIDIA GPUs and MLX kernels for Macs, iPhones, and iPads with sufficient memory. Standard inference libraries generally lack native support for 1.76-bit ternary weights, so the custom matrix-multiplication path determines both compatibility and much of the performance.

Local agents gain a larger working model

PrismML demonstrates Bonsai 2 through a Cline demo for agentic coding and a computer-use loop running locally on an RTX 5090. Both workloads require the model to interpret changing state, select actions, process tool results, and maintain coherence across repeated steps. The release supports several deployment patterns:

• Local coding loops when repositories, tools, and execution environments remain on the workstation;
• Computer-use agents that process screenshots and interface state on-device;
• Document and image analysis with a reported context limit of 262K tokens;
• Hybrid routing that handles frequent or sensitive requests locally and sends selected queries to remote models;
• Resident laptop assistants with a smaller weight-memory requirement than conventional 27B deployments.

Local inference can keep prompts and model outputs on the device, reducing the data sent to a hosted model API. Agent tools, telemetry systems, package managers, and external search services may still create network traffic, so application architecture determines the final privacy boundary.

Memory and task regressions set the limits

• Runtime overhead: The 5.9 GB figure covers weights. Activations, caches, buffers, and the application consume additional memory.
• Long-context memory: Key-value cache usage grows with sequence length, making the advertised 262K-token window impractical on some supported devices.
• Vision accuracy: The reported three-point decline can affect screenshot analysis, visual grounding, and computer-use agents.
• Tool reliability: A 2.17-point agentic decline can compound across multi-step workflows, making retries and validation important.
• Kernel availability: Deployment depends on PrismML’s optimized CUDA, MLX, or WebGPU paths rather than broad support in established inference engines.
• Base-model ceiling: Quantization preserves many capabilities of Qwen3.8 27B along with its underlying limitations.

Compression changes deployment density

A 5.9 GB weight package gives developers access to a larger local model within memory budgets usually associated with smaller parameter counts. In data centers, the same density can support more replicas per GPU or leave more memory for caches and concurrent requests, subject to runtime overhead and workload shape.

PrismML is a Caltech spinout founded with support from Khosla Ventures, Cerberus, and Google, with continuing support from Samsung. Bonsai 2 advances its effort to optimize model capability for fixed memory and power budgets, with production value now depending on independent validation and application-level testing across vision, tool use, and long contexts.