---
title: "PrismML's Ternary Bonsai 2 Runs 262K Context on a 12 GB GPU"
subtitle: "A patched serving stack runs the 27B Ternary Bonsai 2 at the full 262k context window on a single 12 GB consumer GPU, with agentic tool-calling finally fixed."
url: "https://alphasignal.ai/news/prismml-s-ternary-bonsai-2-runs-262k-context-on-a-12-gb-gpu"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-10-07T00:17:36.000Z"
updated: "2026-10-08T13:06:30.434Z"
type: "Model"
topics:
  - "Llms"
  - "Infra"
subtopics:
  - "Long Context"
  - "Inference Optimization"
upvotes: 178
source_url: "https://huggingface.co/CaryPalmer/Ternary-Bonsai-2-27B-262k-GGUF"
comments: 0
truncated: true
---

# PrismML's Ternary Bonsai 2 Runs 262K Context on a 12 GB GPU

> A patched serving stack runs the 27B Ternary Bonsai 2 at the full 262k context window on a single 12 GB consumer GPU, with agentic tool-calling finally fixed.

A community serving package for PrismML’s 27-billion-parameter Ternary Bonsai 2 reports support for the model’s full 262,144-token context window on a 12 GB RTX 4070\. It keeps the key-value cache at `q8_0` precision, reaches 100 generated tokens per second at 32K context, and adds server-side fixes for reasoning and tool-calling APIs.

The package combines the existing 1.58-bit ternary model with a Multi-Token Prediction draft head and 33 llama.cpp patches. The project attributes several reported failures to server defaults, template validation, and tool formatting. All performance and benchmark figures below come from the publisher.

## One model, a rebuilt runtime

The [release page](https://huggingface.co/CaryPalmer/Ternary-Bonsai-2-27B-262k-GGUF) distributes GGUF files for llama.cpp-based inference. The underlying model comes from PrismML’s [base weights](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf), which encode most parameters with three possible values and average about 1.58 bits per weight.

According to the release, all 851 tensors from the base `PTQ1_0` file remain byte-identical. The only model-file additions are 15 tensors for the draft head, which proposes several likely tokens before the main model verifies them. The serving code, launcher, and optional server layer use the MIT license; PrismML’s weights retain the Apache 2.0 license.

## Triple-digit decode at 32K

On an RTX 4070 with 12 GB of VRAM, the patched server reports the following generation rates. Decode speed measures token generation after prompt processing, so these figures do not describe the time required to ingest a long prompt.

__Publisher-reported RTX 4070 decode performance__
| Context length | Patched server | Reference server |
| -------------- | -------------- | ---------------- |
| 32K            | 100 tokens/s   | 47.6 tokens/s    |
| 64K            | 87 tokens/s    | 36.9 tokens/s    |

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Ternary-Bonsai-2-27B-262k-GGUF Model Card](https://huggingface.co/CaryPalmer/Ternary-Bonsai-2-27B-262k-GGUF)
- [bonsai-ada-surgery GitHub repository](https://github.com/professorpalmer/bonsai-ada-surgery)
- [Cary Palmer announcement on X](https://twitter.com/CaryPalmerr/status/2107272801568636943)
- [PrismML Ternary Bonsai 2 27B base weights](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf)
