How does Bonsai 27B fit on a phone?

Bonsai 27B stores its 27.3B weights at 1.125 or 1.71 effective bits each using PrismML's ternary and 1-bit quantization, cutting the model to 3.9 GB or 5.9 GB on disk. That footprint fits in the ~4–6 GB of RAM a modern phone can spare, so the full 27B parameter count runs locally instead of only in a data-center.

How much quality does Bonsai 27B lose from quantization?

Across PrismML's 15-benchmark suite in thinking mode, Bonsai 27B ternary retains 95% of the full-precision baseline (80.5 vs 85.0) and the 1-bit variant retains 90% (76.1). The paper positions the trade-off as a 27B-class model at a footprint smaller than most full-precision 2B models.

How fast is Bonsai 27B on real hardware?

PrismML reports Bonsai 27B at up to 163 tok/s on an NVIDIA RTX 5090 (1-bit) and 134 tok/s ternary, and up to 87 tok/s on an Apple M5 Max (1-bit) with 58 tok/s ternary. Phone throughput is lower but the on-device budget is set by the ~4 GB RAM ceiling, not the FLOPs.

Where can I download Bonsai 27B?

PrismML publishes Bonsai 27B on Hugging Face in one collection (`prism-ml/bonsai-27b`) with MLX 1-bit, MLX 2-bit ternary, GGUF, AWQ-4bit, and unpacked full-precision variants, all under Apache-2.0. A hosted API is also live at together.ai, and the model runs in-browser through the WebGPU kernels Space.

PrismML · 2026-07-14 · major

Bonsai 27B — first 27B-class LLM to run on a phone at ~4 GB

Bonsai 27B is a 27.3B open-weight model distilled from Qwen3.6-27B that fits in ~4 GB of RAM on a phone using 1-bit or ternary weights, has a 262K-token context, and keeps 90–95% of the full-precision baseline on a 15-benchmark suite.

Hugging Face social card for the Bonsai-27B MLX 1-bit model by PrismML

Bonsai 27B is the first 27B-class open-weight model small enough to run on a phone — 3.9 GB of weights, 262K context, Apache-2.0.

Key specs

Parameters	27.3B
Context window	262K tokens
Weights on disk (1 bit)	3.9 GB
Baseline retention (ternary)	95%

Quick facts

Maker	PrismML (Caltech spinout)
License	Apache-2.0
Base model	Qwen3.6-27B
Variants	1-bit (3.9 GB) and ternary 1.71-bit (5.9 GB)
Context window	262K tokens
On-device RAM	~4 GB (1-bit) / ~6 GB (ternary)
Hardware	Apple MLX (Mac/iPhone/iPad), NVIDIA CUDA, WebGPU

What is it?

Bonsai 27B is PrismML's new multimodal flagship: a 27.3B open-weight language model distilled from Qwen3.6-27B and released in ternary and 1-bit quantizations under Apache-2.0. Weights ship as small as 3.9 GB, so the whole 27B parameter count fits in a modern phone's RAM. A vision tower is optional for image inputs, and the context window is 262K tokens.

How does it work?

PrismML compresses each of Bonsai 27B's weights to 1.125 effective bits (1-bit) or 1.71 bits (ternary), which is what turns a 27B model into a 3.9–5.9 GB file. The backbone uses hybrid attention (~75% linear, ~25% full) with SwiGLU MLP, RoPE, and RMSNorm, plus custom MLX, CUDA, AWQ, GGUF, and WebGPU kernels so the compressed weights run natively on Apple Silicon, NVIDIA GPUs, and the browser.

Why does it matter?

A 27B model that fits in ~4 GB of RAM makes an actual frontier-sized brain runnable on a phone or a laptop, not just a demo. PrismML measures 90–95% of the full-precision baseline on their 15-benchmark suite, so the compression is close to lossless. That closes the gap between what a local model can do and what only a server-hosted model used to.

Who is it for?

mobile devs, edge inference builders, anyone deploying local LLMs

Frequently asked questions

How does Bonsai 27B fit on a phone?: Bonsai 27B stores its 27.3B weights at 1.125 or 1.71 effective bits each using PrismML's ternary and 1-bit quantization, cutting the model to 3.9 GB or 5.9 GB on disk. That footprint fits in the ~4–6 GB of RAM a modern phone can spare, so the full 27B parameter count runs locally instead of only in a data-center.
How much quality does Bonsai 27B lose from quantization?: Across PrismML's 15-benchmark suite in thinking mode, Bonsai 27B ternary retains 95% of the full-precision baseline (80.5 vs 85.0) and the 1-bit variant retains 90% (76.1). The paper positions the trade-off as a 27B-class model at a footprint smaller than most full-precision 2B models.
How fast is Bonsai 27B on real hardware?: PrismML reports Bonsai 27B at up to 163 tok/s on an NVIDIA RTX 5090 (1-bit) and 134 tok/s ternary, and up to 87 tok/s on an Apple M5 Max (1-bit) with 58 tok/s ternary. Phone throughput is lower but the on-device budget is set by the ~4 GB RAM ceiling, not the FLOPs.
Where can I download Bonsai 27B?: PrismML publishes Bonsai 27B on Hugging Face in one collection (`prism-ml/bonsai-27b`) with MLX 1-bit, MLX 2-bit ternary, GGUF, AWQ-4bit, and unpacked full-precision variants, all under Apache-2.0. A hosted API is also live at together.ai, and the model runs in-browser through the WebGPU kernels Space.

Try it

huggingface.co/collections/prism-ml/bonsai-27b (MLX / GGUF / AWQ / WebGPU)