What hardware do I need to run Kimi K3 on WASTE?

WASTE runs Kimi K3 on a machine with at least 29 GB of RAM (64 GB recommended), roughly 982 GiB of internal NVMe storage for the weights, and a modern x86 or Apple Silicon CPU. External USB SSDs cause severe slowdown — the streaming design relies on the low latency of an internal PCIe NVMe drive.

How fast is Kimi K3 on WASTE compared to a GPU?

WASTE decodes Kimi K3 at 0.49–0.54 tokens per second on a 64 GB MacBook Pro — slow enough to feel like a batch job, not a chat. The point is not speed but that the full 2.78-trillion-parameter model runs at all on consumer hardware, without distillation, pruning, or a rented GPU.

How does WASTE differ from llama.cpp or vLLM?

WASTE targets a different problem: fitting a giant mixture-of-experts model on a laptop. It keeps the trunk in RAM and streams individual experts from disk on demand, using leftover RAM as a bounded expert cache. llama.cpp and vLLM both assume the whole model fits in RAM or GPU memory, so they cannot serve K3 on a 64 GB machine at all.

Is WASTE limited to Kimi K3?

WASTE also supports Kimi-Linear at 48 B parameters, which decodes at 10.7 tokens per second in an 8 GB RAM budget. The engine is designed around large sparse mixture-of-experts models with 3-bit residual quantization, so any model that fits that shape should port cleanly, but K3 and Kimi-Linear are the two first-class targets today.

SQLite AI · 2026-07-30 · major

WASTE — run 2.78T Kimi K3 on a 64GB laptop by streaming from NVMe

SQLite AI's WASTE (Weight-Aware Streaming Tensor Engine) is a dependency-free C inference engine that runs the full 2.78-trillion-parameter Kimi K3 on 64 GB of RAM by streaming experts from NVMe at 0.49–0.54 tokens per second.

GitHub Open Graph card for sqliteai/waste — the WASTE inference engine

SQLite AI's WASTE streams Kimi K3's experts from an NVMe SSD so the full 2.78-trillion-parameter model runs on a 64 GB laptop.

Key specs

GitHub stars	378
Decode speed	0.49–0.54 tok/s
Ram required	29 GB

Quick facts

Maker	SQLite AI
License	Apache-2.0
Language	C, no runtime deps
Target model	Kimi K3 (2.78T params)
Minimum RAM	29 GB (64 GB recommended)
Storage	982 GiB internal NVMe
Latest release	v0.6.1 on 2026-07-31

What is it?

WASTE (Weight-Aware Streaming Tensor Engine) is a small C library from SQLite AI that runs Kimi K3, Moonshot's 2.78-trillion-parameter mixture-of-experts model, on consumer hardware. Instead of loading the whole model into RAM, WASTE keeps only the shared trunk in memory and streams the active experts from an internal NVMe SSD.

How does it work?

WASTE stores expert weights on disk using 3-bit residual vector quantization. For each token the router picks a small set of experts, WASTE reads only those experts from NVMe, runs them through a bounded RAM cache, and discards them. The engine has no runtime dependencies beyond libc and pthreads, ships CLI, HTTP server, and embeddable-API interfaces, and includes a vision tower so K3 handles image input as well as text.

Why does it matter?

Before WASTE, running the full Kimi K3 checkpoint required a multi-GPU server with hundreds of gigabytes of RAM — out of reach for individual developers and small labs. WASTE turns a 64 GB laptop into a real (if slow) K3 host at about half a token per second, opening a path for local experimentation with frontier open-weights models where memory, not compute, was the ceiling.

Who is it for?

researchers and hobbyists running open-weights frontier MoE models on personal hardware

Frequently asked questions

What hardware do I need to run Kimi K3 on WASTE?: WASTE runs Kimi K3 on a machine with at least 29 GB of RAM (64 GB recommended), roughly 982 GiB of internal NVMe storage for the weights, and a modern x86 or Apple Silicon CPU. External USB SSDs cause severe slowdown — the streaming design relies on the low latency of an internal PCIe NVMe drive.
How fast is Kimi K3 on WASTE compared to a GPU?: WASTE decodes Kimi K3 at 0.49–0.54 tokens per second on a 64 GB MacBook Pro — slow enough to feel like a batch job, not a chat. The point is not speed but that the full 2.78-trillion-parameter model runs at all on consumer hardware, without distillation, pruning, or a rented GPU.
How does WASTE differ from llama.cpp or vLLM?: WASTE targets a different problem: fitting a giant mixture-of-experts model on a laptop. It keeps the trunk in RAM and streams individual experts from disk on demand, using leftover RAM as a bounded expert cache. llama.cpp and vLLM both assume the whole model fits in RAM or GPU memory, so they cannot serve K3 on a 64 GB machine at all.
Is WASTE limited to Kimi K3?: WASTE also supports Kimi-Linear at 48 B parameters, which decodes at 10.7 tokens per second in an 8 GB RAM budget. The engine is designed around large sparse mixture-of-experts models with 3-bit residual quantization, so any model that fits that shape should port cleanly, but K3 and Kimi-Linear are the two first-class targets today.

Try it

git clone https://github.com/sqliteai/waste