# Ternary Bonsai — 1.58-Bit 8B Runs at 82 tok/s on M4 Pro in 1.75 GB

> 1.58-bit ternary weights fit an 8B model in 1.75 GB and run fast enough for interactive use on iPhones and MacBooks.

- Org: PrismML
- Date: 2026-04-16
- Category: model, repo
- Source: https://prismml.com/news/ternary-bonsai
- Canonical page: https://ai-tldr.dev/releases/prismml-ternary-bonsai/

## In short

PrismML's Ternary Bonsai squeezes an 8B model into 1.75 GB using 1.58-bit ternary weights — 9× smaller than FP16, 82 tok/s on M4 Pro, 27 tok/s on iPhone 17 Pro Max, 75.5 avg benchmark. Apache 2.0, GGUF and MLX variants on HuggingFace.

## What it is

Ternary Bonsai is a family of language models from PrismML where every weight is one of three values: -1, 0, or +1. Available in 8B, 4B, and 1.7B sizes, the 8B variant occupies 1.75 GB — about 9× smaller than a standard FP16 8B model. All models are Apache 2.0 and distributed as GGUF and MLX (Apple Silicon) variants on HuggingFace. PrismML also ships a pre-built native runner (Bonsai-demo) supporting Metal, CUDA, ROCm, Vulkan, and CPU backends.

## How it works

Ternary quantization constrains weights to {-1, 0, +1}, replacing floating-point multiplies with additions and subtractions. This slashes storage and reduces compute cost on hardware with efficient integer pipelines — especially Apple Silicon's Neural Engine. The 8B model scores 75.5 averaged across MMLU Redux, MuSR, GSM8K, HumanEval+, IFEval, and BFCLv3, competitive with other open 8B models. PrismML also ships a 1-bit Bonsai family (weights in {-1, +1}) which trades benchmark score for even smaller footprint.

## Why it matters

Fitting an 8B model in 1.75 GB means it runs in the RAM headroom of current iPhones and RAM-constrained laptops without any extra quantization step. At 82 tok/s on M4 Pro and 27 tok/s on iPhone 17 Pro Max the throughput crosses the interactive threshold — comparable to what llama.cpp achieves with Q4 quants but at a smaller footprint. The Apache 2.0 license and ready-to-run GGUF files lower the barrier to entry for on-device deployment.

## Who it's for

Developers targeting on-device inference on Apple Silicon or iPhones; self-hosters with memory-constrained hardware.

## Try it

huggingface.co/collections/prism-ml/ternary-bonsai — GGUF and MLX variants

## Links

- [Announcement](https://prismml.com/news/ternary-bonsai)
- [Repo](https://github.com/PrismML-Eng/Bonsai-demo)
- [HF Collection](https://huggingface.co/collections/prism-ml/ternary-bonsai)
- [Demo](https://huggingface.co/spaces/webml-community/bonsai-ternary-webgpu)