hyperionaidev · 2026-11-12 · notable
smolmaxxing — a 16B MoE model runs on a 1 GB TV box and a 2013 iPad
In short
smolmaxxing is an MIT llama.cpp patch for 1 GB devices. It runs Qwen3.5-2B at 7.7 tokens/s and the 16B Ling-mini-2.0 MoE at 5.1 tokens/s on a Xiaomi TV box, and 1-bit Bonsai-1.7B at 24–26 tokens/s on a 2013 iPad Air.
hyperionaidev · 2026-11-12 · 2 sources checked
Per-chip kernels and experts paged from flash let 16B models run on devices with about 1 GB of RAM.
Key specs
| 16 b mo e on tv box | 5.1 tok/s |
|---|---|
| Usable ram | ~1 GB |
What is it?
smolmaxxing runs language models from 260K to 16B parameters on a Xiaomi TV Box S (3rd Gen) with about 1 GB of usable RAM and a jailbroken 2013 iPad Air with about 700 MB. The project is a patch on llama.cpp, and it says every number it reports was measured on the device itself.
How does it work?
Custom kernels are written per chip, including a 1-bit lookup-table kernel, repacked 2-bit and ternary formats for the Cortex-A55, and Apple A7 versions without dot-product instructions. Mixture-of-experts models keep popular experts in RAM and read the rest from flash, so 8B and 16B MoE models fit in about 1 GB. Rarely used vocabulary rows are dropped, and on the TV box a headless mode stops the launcher while a model runs.
Why does it matter?
Old and cheap hardware can run useful local models offline, without a GPU or a new phone. The project's notes on which tricks helped, and which did not, are a practical guide for anyone working on memory-limited inference. Models need the patched llama.cpp, because stock llama.cpp cannot load them.
Who is it for?
edge-AI hackers and local-LLM tinkerers
Try it
https://github.com/hyperionaidev/smolmaxxing/blob/main/docs/tv-box-setup.md