# smolmaxxing — a 16B MoE model runs on a 1 GB TV box and a 2013 iPad

> Per-chip kernels and experts paged from flash let 16B models run on devices with about 1 GB of RAM.

- Org: hyperionaidev
- Date: 2026-11-12
- Category: repo, tool
- Source: https://github.com/hyperionaidev/smolmaxxing
- Canonical page: https://ai-tldr.dev/releases/hyperionaidev-smolmaxxing/

## In short

smolmaxxing is an MIT llama.cpp patch for 1 GB devices. It runs Qwen3.5-2B at 7.7 tokens/s and the 16B Ling-mini-2.0 MoE at 5.1 tokens/s on a Xiaomi TV box, and 1-bit Bonsai-1.7B at 24–26 tokens/s on a 2013 iPad Air.

## What it is

smolmaxxing runs language models from 260K to 16B parameters on a Xiaomi TV Box S (3rd Gen) with about 1 GB of usable RAM and a jailbroken 2013 iPad Air with about 700 MB. The project is a patch on llama.cpp, and it says every number it reports was measured on the device itself.

## How it works

Custom kernels are written per chip, including a 1-bit lookup-table kernel, repacked 2-bit and ternary formats for the Cortex-A55, and Apple A7 versions without dot-product instructions. Mixture-of-experts models keep popular experts in RAM and read the rest from flash, so 8B and 16B MoE models fit in about 1 GB. Rarely used vocabulary rows are dropped, and on the TV box a headless mode stops the launcher while a model runs.

## Why it matters

Old and cheap hardware can run useful local models offline, without a GPU or a new phone. The project's notes on which tricks helped, and which did not, are a practical guide for anyone working on memory-limited inference. Models need the patched llama.cpp, because stock llama.cpp cannot load them.

## Who it's for

edge-AI hackers and local-LLM tinkerers

## Try it

https://github.com/hyperionaidev/smolmaxxing/blob/main/docs/tv-box-setup.md

## Links

- [GitHub Repo](https://github.com/hyperionaidev/smolmaxxing)
- [TV Box Setup](https://github.com/hyperionaidev/smolmaxxing/blob/main/docs/tv-box-setup.md)
- [HN Discussion](https://news.ycombinator.com/item?id=50468888)