leonickson1 · 2026-08-03 · notable
Swiftlet — 80B Qwen runs in 4.3 GB on a Mac, 35B on an iPhone
Swiftlet is an Apache-2.0 Swift and Metal runtime that streams routed MoE experts from disk on demand. It runs Qwen3-Next and Qwen3.5/3.6 hybrid MoE models on Apple hardware — 80B in 4.3 GB peak RAM on a Mac, 35B in about 2.5 GB on an iPhone 17.
Swift and Metal runtime that runs Qwen 35B on an iPhone and 80B on a Mac by streaming MoE experts from local storage.
What is it?
Swiftlet is an Apache-2.0 project from developer leonickson1 that hosts Qwen3-Next and Qwen3.5/3.6 hybrid MoE models on Apple devices. The trick: only a small dense core sits in RAM, and the routed mixture-of-experts weights stream from local storage as tokens need them.
How does it work?
Every token routes through a small subset of experts, so Swiftlet only needs the active experts in memory. The routed weights stream from disk on demand while a small dense core stays resident — Swiftlet drops an 80B footprint from tens of gigabytes of RAM to about 4.3 GB peak, at the cost of extra storage I/O.
Why does it matter?
MoE models of this size have needed workstations or servers. Swiftlet makes a 35B model feasible on an iPhone 17 at about 2.5 GB peak RAM and ~1 token/second, and 80B feasible on a consumer Mac at 4.3 GB RAM and 4.5–5 tokens/second decode. No cloud call, no data leaves the device.
Who is it for?
iOS and macOS developers, on-device AI hobbyists
Try it
git clone https://github.com/leonickson1/Swiftlet