WARP (SQLite AI)

// Overview

WARP — Weight-Aware Runtime and Paging, released under the name WASTE — is an inference engine written in C with no third-party runtime dependencies. It keeps the dense trunk of a Mixture-of-Experts model in memory, streams the experts a token actually activates directly from disk, and uses whatever RAM is left as a bounded expert cache. That trade turns the memory requirement of a very large model into a storage-bandwidth problem, which is why the project targets NVMe rather than more DRAM.

The README reports the complete 2.78-trillion-parameter Kimi K3 running on a 64 GB MacBook Pro at about 0.6 tokens per second, the 313-billion-parameter GLM-5.3-Flash (text and images) at about 3.9, and the 48-billion-parameter Kimi-Linear at roughly 14–17. The DeepSeek-V3 family is supported as well. The project is deliberately narrow: it exists to find out how far local inference can be pushed when weights live mostly on fast storage instead of RAM, and it is candid that humans set the direction while the C code itself is largely written by LLMs.

WARP ships as a command-line binary, an OpenAI-compatible HTTP server and an embeddable C library, so it can sit inside another application rather than beside it. Models are converted into a single `.waste` container with the Python tooling in the repository, and vision towers are supported for both Kimi K3 and the GLM models. It is Apache-2.0 licensed and the maintainers state the project will always remain open source under a permissive licence.

// What it does

• Streams the routed Mixture-of-Experts weights from NVMe on demand while the dense trunk stays resident in RAM
• Bounded expert cache uses the memory you have left rather than requiring the whole model to fit
• Written in C with no third-party runtime dependencies, and embeddable as a library in another program
• Runs trillion-parameter models on a laptop: the full 2.78T Kimi K3 at ~0.6 tok/s on a 64 GB MacBook Pro
• Multimodal — vision tower support for both Kimi K3 and GLM, alongside the DeepSeek-V3 family
• OpenAI-compatible HTTP server for dropping it behind an existing client

// Getting started

WARP builds with a plain make and runs against a single .waste model container. There is nothing else to install.

git clone https://github.com/sqliteai/warp
cd warp
make
make check

./waste run ~/models/k3.waste "What is the capital of Italy?"

./waste chat ~/models/glm53.waste

python3 -m serve ~/models/k3.waste --port 8000

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Run a frontier-size Mixture-of-Experts model locally on a machine whose RAM is far smaller than the weights
• Trade storage bandwidth for memory when adding DRAM or renting GPUs is not an option
• Embed local inference directly inside a C application without pulling in a runtime stack
• Experiment with how far weight paging can be pushed before latency becomes unusable

// How WARP (SQLite AI) compares

WARP (SQLite AI) alongside other open-source local runtimes tools AI/TLDR tracks, ranked by GitHub stars.