PrismML · 2026-09-17 · major
Bonsai 2 27B — ternary model keeps 98.2% of full precision in 5.9 GB
Bonsai 2 27B is an Apache-2.0 ternary model built from Qwen3.8 27B. It stores every weight as -1, 0 or +1 at 1.76 effective bits, scores 83.9 overall, and keeps 98.2% of the full-precision model with a 262K-token context.
.png)
A 27B model squeezed into ternary weights that gives up 1.5 points of benchmark score for a 9x smaller download.
Key specs
| Footprint | 5.9 GB |
|---|---|
| Retention | 98.2% |
Quick facts
| Maker | PrismML |
|---|---|
| Base model | Qwen3.8 27B |
| Weights | Ternary {-1, 0, +1}, 1.76 effective bits |
| Context window | 262K tokens |
| License | Apache-2.0 |
| Runs on | CUDA (NVIDIA) and MLX (Apple) |
| Speed | 143 tok/s on RTX 5090, 46.8 tok/s on M5 Max |
Benchmarks
What is it?
Bonsai 2 27B moves to a newer base model, Qwen3.8 27B, and pushes capability retention to 98.2% — the first Bonsai 27B held 90-95%. Instead of storing each weight as a floating-point number, PrismML stores it as one of three values: -1, 0 or +1. With FP16 group-wise scaling on top, that averages 1.76 bits per weight, small enough to keep a 27B-class model resident in memory on a laptop or a phone.
How does it work?
Ternary quantization is the mechanism behind Bonsai 2 27B. Each weight collapses to one of three states, and a shared FP16 scale factor per group of weights restores the range the three states cannot express on their own. Matrix multiplication then becomes mostly addition and subtraction rather than floating-point multiplication, which is why the model reaches 143 tokens per second on an RTX 5090. PrismML ships kernels for CUDA on NVIDIA cards and MLX on Apple silicon.
Why does it matter?
Running a 27B model locally normally means a 50+ GB download and a workstation GPU. At 5.9 GB, Bonsai 2 27B fits on hardware people already own, so private, offline inference stops being a datacenter question. The Apache-2.0 license means companies can ship it inside a product without a usage agreement, and the 262K-token context is long enough for whole-codebase or long-document work on-device.
Who is it for?
on-device and edge developers
Frequently asked questions
- How is Bonsai 2 27B different from the first Bonsai 27B?
- Bonsai 2 27B is distilled from Qwen3.8 27B, while the original Bonsai 27B was built on Qwen3.6 27B. The retention figure is the clearest gap: Bonsai 2 keeps 98.2% of its full-precision baseline against the 90-95% band PrismML reported for the first release. PrismML also cites gains in reasoning, coding, vision and agentic work.
- Can I use Bonsai 2 27B in a commercial product?
- Yes. PrismML publishes Bonsai 2 27B under the Apache 2.0 license, which permits commercial use, modification and redistribution without a separate agreement or revenue threshold. The weights are on Hugging Face in the Bonsai 2 collection, with GGUF builds for CUDA machines and a 2-bit MLX build for Apple silicon.
- Where does Bonsai 2 27B lose the most against full precision?
- Agentic and tool calling is the weakest area for Bonsai 2 27B, at 77.57 against 79.74 for full-precision Qwen3.8 27B. Knowledge and reasoning drops to 83.95 from 86.66, and vision to 78.59 from 81.64. It moves the other way on instruction following, scoring 82.66 against 81.25 for the uncompressed model.
- Can Bonsai 2 27B run in a web browser?
- The Bonsai 2 collection on Hugging Face includes a Space called Ternary Bonsai 2 WebGPU Kernels, which runs the model locally in a browser through WebGPU. For native use, PrismML ships CUDA kernels for NVIDIA GPUs and MLX builds for Mac, iPhone and iPad rather than requiring a browser runtime.
Try it
prism-ml/Ternary-Bonsai-2-27B-gguf