How much RAM do I need to run Slotstream?

Slotstream needs at least 8 GB of RAM on an Apple Silicon Mac, but speed scales with memory: about 3 tokens per second at 8 GB, about 9 at 32 GB, and about 12 on a 48 GB M5 Pro. You also need roughly 110 GB of free disk space for the model, so a 512 GB Mac is the practical floor.

Which models does Slotstream support?

Slotstream supports one model today: qwen3.8-flash-next:4bit. The engine is built around that model's exact geometry, including its 48 layers and 512 experts per layer with 10 routed per token, so other checkpoints will not load. The README lists no other supported models.

Is Slotstream free and open source?

Slotstream is free and open source under the MIT license, and the vendored mlx-swift-lm code it builds on is MIT too. The Qwen3.8-Flash-Next weights are separate: Hugging Face lists them under the qwen-community-1.0 license, so what you may do commercially depends on Qwen's terms, not Slotstream's.

Can I turn the speculative decoding off?

Yes. Slotstream v0.2.0 adds an --mtp flag with auto, on and off settings to its run, serve and doctor commands. The default, auto, switches the multi-token-prediction head on only when it is worth the roughly 1.6 GB of extra cache it needs. Without the converted head file, Slotstream behaves exactly as earlier versions did.

Carlos Galarza · 2026-09-01 · major

Slotstream v0.2.0 — a 104GB model on a 48GB Mac, plus speculative decoding

Slotstream runs Qwen3.8-Flash-Next, a 125B mixture-of-experts model that takes 104 GB on disk, on Apple Silicon Macs with far less RAM by streaming experts from SSD. Version 0.2.0 adds speculative decoding.

GitHub card for Slotstream, a Swift engine that streams model experts from SSD on Macs

Slotstream streams a 125B model's experts off SSD so a Mac with 48 GB of RAM can run a 104 GB checkpoint.

Key specs

Model on disk	104 GB
Peak ram used	32 GB

Quick facts

Version	v0.2.0 (v0.2.1 patch on Sep 2)
License	MIT
Platform	Apple Silicon, macOS 14+
Supported model	qwen3.8-flash-next:4bit only
Decode speed	~12 tok/s on a 48 GB M5 Pro
Disk needed	~110 GB free
API	Ollama and OpenAI SDK compatible

What is it?

Slotstream v0.2.0 adds speculative decoding to a Swift engine that runs Qwen3.8-Flash-Next on Apple Silicon Macs that do not have enough RAM to hold the model. The checkpoint is 125B parameters and takes 104 GB on disk at 4-bit. Carlos Galarza shipped the first version on 2026-08-28, and it is one Swift binary with no Python.

How does it work?

The trick is keeping only part of the model in memory. A 3.8 GB dense trunk stays resident while the 68 GB of routed experts sit on disk, 512 per layer with 10 active per token. Rather than memory-mapping the whole file, Slotstream uses pread to pull experts into a fixed cache pool shared across all 48 layers, so hot layers borrow slots from cold ones. Version 0.2.0 adds a draft step using the multi-token-prediction head found in the checkpoint, which the release measures at an 85.8% first-draft accept rate.

Why does it matter?

A 125B open-weight model normally means a workstation or a rented GPU. Slotstream puts one on a laptop people already own, at about 12 tokens per second on a 48 GB M5 Pro and roughly 9 on 32 GB. Because it answers the same chat and generate requests that Ollama clients and OpenAI SDKs send, existing local-AI tools can point at it without code changes.

Who is it for?

Mac users running local models

Frequently asked questions

How much RAM do I need to run Slotstream?: Slotstream needs at least 8 GB of RAM on an Apple Silicon Mac, but speed scales with memory: about 3 tokens per second at 8 GB, about 9 at 32 GB, and about 12 on a 48 GB M5 Pro. You also need roughly 110 GB of free disk space for the model, so a 512 GB Mac is the practical floor.
Which models does Slotstream support?: Slotstream supports one model today: qwen3.8-flash-next:4bit. The engine is built around that model's exact geometry, including its 48 layers and 512 experts per layer with 10 routed per token, so other checkpoints will not load. The README lists no other supported models.
Is Slotstream free and open source?: Slotstream is free and open source under the MIT license, and the vendored mlx-swift-lm code it builds on is MIT too. The Qwen3.8-Flash-Next weights are separate: Hugging Face lists them under the qwen-community-1.0 license, so what you may do commercially depends on Qwen's terms, not Slotstream's.
Can I turn the speculative decoding off?: Yes. Slotstream v0.2.0 adds an --mtp flag with auto, on and off settings to its run, serve and doctor commands. The default, auto, switches the multi-token-prediction head on only when it is worth the roughly 1.6 GB of extra cache it needs. Without the converted head file, Slotstream behaves exactly as earlier versions did.

Try it

curl -fsSL https://raw.githubusercontent.com/carloslfu/slotstream/main/install.sh | sh