Deltafin

// Overview

Deltafin is a single compiled Rust binary that runs the complete, unmodified Kimi K3 — 2.8 trillion parameters in a Mixture-of-Experts layout — on one consumer machine. The design choice that defines it is refusing to shrink the model: all 16 experts still process every token, so nothing is pruned, distilled or quantized away to make the weights fit. Instead of fitting the model into memory, Deltafin leaves the expert weights on storage and streams each expert in only when the router asks for it, using router tracing and expert prefetch to hide the read latency. Tokenization is native to the binary, so there is no Python at runtime.

The ARGODRIVE build maintained by Argonaut Labs is a fork of gavamedia's original Deltafin, adding the multi-SSD storage work and a published benchmark package. On an M5 Max MacBook Pro with 128 GB of RAM and 1.45 TB of weights spread across four SSDs, it reports steady decode of 1.00 tokens per second on a 512-token answer and 1.13 tok/s on a 128-token one. Drive count is the main lever: one drive reaches roughly 52% of the four-drive speed, two 73% and three 90%. An optional smaller draft model called DSpark speeds up decoding, with K3 itself verifying every token DSpark proposes, so the output is unchanged.

Deltafin runs on macOS arm64, Linux x86-64 and Linux aarch64, with MPS/Metal, CUDA and CPU acceleration paths. Besides an interactive chat mode it exposes an OpenAI-compatible REST server for chat completions and raw completions, so local coding agents and chat clients can point at it directly. The project is independent of Moonshot AI; Deltafin's own code is MIT-licensed while the K3 weights keep their upstream terms.

// What it does

• Runs the full unmodified 2.8T-parameter Kimi K3 with all 16 experts active per token — no pruning, distillation or quality trade to fit the machine
• Streams expert weights on demand from one or more SSDs, with router tracing and expert prefetch used to hide storage latency
• Measured multi-drive scaling: one SSD gives about 52% of four-drive speed, two 73% and three 90%
• Optional DSpark draft model (about 4.49 GiB when admitted at runtime) accelerates decoding, with K3 verifying every proposed token
• Single compiled Rust binary with C-ABI providers and native tokenization — no Python needed at runtime
• OpenAI-compatible REST server (`deltafin serve`) for chat completions and raw completions, so local agents and chat UIs can connect
• MPS/Metal, CUDA and CPU acceleration paths across macOS arm64, Linux x86-64 and Linux aarch64

// Getting started

You need a supported platform (macOS arm64, Linux x86-64 or Linux aarch64), a Rust toolchain to build, and a lot of disk: about 1.7 TB for the full model download, or 215 GB minimum in streaming mode, plus 6.635 GiB for DSpark and an optional 4.337 GiB for Qwen.

git clone https://github.com/gavamedia/deltafin.git
cd deltafin
cargo build --locked --release

./target/release/deltafin setup --full

./target/release/deltafin run --chat
./target/release/deltafin run --prompt "Your question here" --stats

./target/release/deltafin serve --host 127.0.0.1 --port 8000

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Run a frontier-scale open-weight model privately on hardware you own instead of renting it behind an API
• Point a local coding agent or chat client at a 2.8T model through an OpenAI-compatible endpoint on localhost
• Trade throughput for capability on batch or overnight work, where roughly 1 token per second from Kimi K3 is acceptable
• Study how far SSD bandwidth and drive striping can substitute for RAM when serving a very large Mixture-of-Experts model

// How Deltafin compares

Deltafin alongside other open-source local runtimes tools AI/TLDR tracks, ranked by GitHub stars.