How much RAM does Colibri need to run GLM-5.2?

Colibri keeps only GLM-5.2's dense layers resident — about 9.9 GB at int4 — while the 21,504 expert modules stream on demand from NVMe. Total system memory around 25 GB is enough to run the 744B mixture-of-experts model end to end, though a warm expert cache is what keeps speed usable.

How fast is Colibri on a laptop?

Colibri is honest that it is not fast: 0.05–0.1 tokens per second cold on the author's WSL2 baseline with 12 cores and 25 GB RAM. Warm speculative decoding lifts that to 2.2–2.8 tokens per forward pass, and community measurements report roughly 1.06 tok/s on an Apple M5 Max and 0.37 tok/s on a Framework 13.

What license and dependencies does Colibri have?

Colibri ships under Apache-2.0 and is written in pure C with zero runtime dependencies — no CUDA, no PyTorch, no BLAS library. The whole engine is roughly 1,300 lines, plus a small setup script and an OpenAI-compatible HTTP gateway that lets standard clients talk to it.

How do I try Colibri?

Clone JustVugg/colibri, run `cd c && ./setup.sh`, then point Colibri at the pre-converted int4 weights on Hugging Face with `COLI_MODEL=/path/to/GLM-5.2-colibri-int4 ./coli chat`. `./coli serve` starts an OpenAI-compatible API server; `./coli bench` runs HellaSwag, ARC, and MMLU.

JustVugg · 2026-07-09 · major

Colibri — pure-C engine runs GLM-5.2 744B on a 25 GB laptop

Colibri is a 1,300-line C inference engine that keeps GLM-5.2's dense layers in 9.9 GB of RAM and streams the 744B model's routed experts from disk on demand, so the frontier MoE answers on a 25 GB laptop at 0.05–0.1 tok/s cold.

GitHub social preview for JustVugg/colibri, a pure-C inference engine for GLM-5.2

Colibri streams GLM-5.2's experts from disk so a 744B frontier MoE runs on a 25 GB laptop.

Key specs

GitHub stars	1,621
Show hn points	769
Ram	~25 GB
Disk	~370 GB

Quick facts

Maker	JustVugg (independent)
License	Apache-2.0
Language	Pure C, zero runtime deps
Model supported	GLM-5.2 (744B, ~40B active)
RAM required	~9.9 GB resident (int4)
Disk required	~370 GB for pre-converted weights
Speed (WSL2, 25 GB RAM)	0.05–0.1 tok/s cold; ~1.06 tok/s on Apple M5 Max

What is it?

Colibri is a new open-source inference engine that runs the full 744B parameter GLM-5.2 mixture-of-experts model on a machine with about 25 GB of RAM. The whole engine is roughly 1,300 lines of pure C with no CUDA and no PyTorch. Author JustVugg posted it to Show HN on July 9 and it hit 769 points, taking the repo to more than 1,600 stars in a day.

How does it work?

The engine keeps GLM-5.2's dense layers — about 9.9 GB at int4 — resident in RAM, and streams the model's 21,504 expert modules from NVMe storage as the router picks them per token. A native multi-token prediction speculative decoder and a compressed KV-cache that persists across sessions cut the cost of long conversations. An OpenAI-compatible HTTP gateway lets existing clients talk to the engine.

Why does it matter?

Frontier-scale open models like GLM-5.2 normally need multi-GPU rigs or a datacenter. Colibri makes the 744B model answer correctly on hardware that costs less than an H100's fan — cold speed is 0.05–0.1 tok/s, warm speculation is 2.2–2.8 tokens per forward pass, and users report up to 1.06 tok/s on an Apple M5 Max. That turns 'run frontier weights locally' from a lab exercise into something a curious developer can do on the laptop they already own.

Who is it for?

Local-inference tinkerers, cost-sensitive developers, and anyone who wants to explore GLM-5.2 without a GPU rig.

Frequently asked questions

How much RAM does Colibri need to run GLM-5.2?: Colibri keeps only GLM-5.2's dense layers resident — about 9.9 GB at int4 — while the 21,504 expert modules stream on demand from NVMe. Total system memory around 25 GB is enough to run the 744B mixture-of-experts model end to end, though a warm expert cache is what keeps speed usable.
How fast is Colibri on a laptop?: Colibri is honest that it is not fast: 0.05–0.1 tokens per second cold on the author's WSL2 baseline with 12 cores and 25 GB RAM. Warm speculative decoding lifts that to 2.2–2.8 tokens per forward pass, and community measurements report roughly 1.06 tok/s on an Apple M5 Max and 0.37 tok/s on a Framework 13.
What license and dependencies does Colibri have?: Colibri ships under Apache-2.0 and is written in pure C with zero runtime dependencies — no CUDA, no PyTorch, no BLAS library. The whole engine is roughly 1,300 lines, plus a small setup script and an OpenAI-compatible HTTP gateway that lets standard clients talk to it.
How do I try Colibri?: Clone JustVugg/colibri, run `cd c && ./setup.sh`, then point Colibri at the pre-converted int4 weights on Hugging Face with `COLI_MODEL=/path/to/GLM-5.2-colibri-int4 ./coli chat`. `./coli serve` starts an OpenAI-compatible API server; `./coli bench` runs HellaSwag, ARC, and MMLU.

Try it

git clone https://github.com/JustVugg/colibri && cd colibri/c && ./setup.sh