# smolmaxxing

> A llama.cpp patch that runs LLMs from 260K to 16B parameters on 1 GB devices: an Android TV box and a 2013 iPad Air

- Category: Models & Inference / Local Runtimes
- Language: C
- Licence: MIT
- Repository: https://github.com/hyperionaidev/smolmaxxing
- Canonical page: https://ai-tldr.dev/tools/smolmaxxing/
- Updated: 2026-11-12

## In short

smolmaxxing is a patch for llama.cpp that tries to run the biggest language model possible on the smallest hardware you own. It sits in the Local Runtimes category of the Models & Inference tools and is open-source, written in C and licensed MIT. To get started: connect to the box and build the patched llama.cpp.

## Overview

smolmaxxing is a patch for llama.cpp that tries to run the biggest language model possible on the smallest hardware you own. It currently targets two devices with about 1 GB of RAM: a Xiaomi TV Box S (3rd Gen) with four Cortex-A55 cores and about 1 GB usable, running Android 14 without root, and a 2013 iPad Air with a two-core Apple A7 and about 700 MB usable, jailbroken on iOS 12.5.7. The project says every number it publishes was measured on the device itself.

The gains come from several tricks stacked together. Custom kernels are written per chip: a 1-bit lookup-table kernel, repacked 2-bit and ternary formats for the Cortex-A55, and versions for the Apple A7, which has no dot-product instructions. Memory use drops by removing rarely used vocabulary rows, and on the TV box a headless mode stops the launcher and Google services while a model runs. For mixture-of-experts models, popular experts stay in RAM and the rest are read from flash on demand, which lets 8B and 16B MoE models run in about 1 GB; dense models larger than RAM are streamed from flash.

The README reports Qwen3.5-2B at 7.7 tokens/s and the 16B Ling-mini-2.0 MoE at 5.1 tokens/s on the TV box, and the 1-bit Bonsai-1.7B at 24–26 tokens/s and Ling-mini-2.0 at 2.6–4.5 tokens/s on the iPad. Models need the patched llama.cpp; stock llama.cpp cannot load them. The code is MIT-licensed and the model weights keep their own licenses. Phones, single-board computers and routers are listed as planned devices.

## What it does

- Per-chip kernels: a 1-bit lookup-table kernel, repacked 2-bit and ternary formats for Cortex-A55, and Apple A7 paths without dot-product instructions
- Mixture-of-experts paging from flash, keeping popular experts in RAM, so 8B and 16B MoE models run in about 1 GB
- Streaming of dense models that are larger than RAM
- Vocabulary pruning that drops rarely used rows of the model's output layer to save memory
- Headless mode on the TV box that stops the launcher and Google services while a model runs
- A browser chat UI served from the TV box with live speed, model and config panels

## Getting started

The TV-box path needs network ADB enabled on the box and adb, zig, cmake, ninja, python3 and uv on your computer. Everything on the box lives in /data/local/tmp/llm. The iPad path is in ipad-air/README.md.

1. Connect to the box and build the patched llama.cpp

   Clone llama.cpp at the pinned commit, apply the project's patch and cross-compile it for the box (the full cmake flags and the adb push commands are in docs/tv-box-setup.md).

```bash
export ADB_SERIAL=<tv-ip>:5555
adb connect $ADB_SERIAL
git clone https://github.com/ggml-org/llama.cpp third_party/llama.cpp
cd third_party/llama.cpp && git checkout 8642541 && git apply ../../patches/llama.cpp-v0.8.0-tvbox.patch
```

2. Get a model

   Push a ready-made file from docs/models.md, or rebuild one from its Hugging Face sources directly onto the box.

```bash
tools/rebuild-models.sh 2b
```

3. Chat from any device on your network

   Start the 2B model and open http://<tv-ip>:8090 in a browser. After MoE models, turn headless mode off on the TV to get the launcher back.

```bash
tv-llm-ui/start.sh
tv-llm-ui/start.sh stop
sh /data/local/tmp/llm/tv-llm-mode.sh off
```

## When to use it

- Turn a cheap Android TV box into a small LLM server for your home network
- Give an old iPad a second life as an offline chat device
- Study which memory and kernel tricks matter most when RAM, not compute, is the limit
- Run a 16B mixture-of-experts model on hardware with about 1 GB of usable RAM

## Version history

- 2026-11-12: smolmaxxing reached the Hacker News front page, showing Qwen3.5-2B at 7.7 tokens/s and a 16B MoE model at 5.1 tokens/s on a Xiaomi TV box with about 1 GB of usable RAM.