Overview
MTPLX is a native Mac app and a command line for running local language models on Apple Silicon. Instead of plain one-token-at-a-time decoding, it uses the model's own multi-token prediction (MTP) heads: the model drafts several tokens ahead of itself, one batched forward pass verifies the draft, and tokens are committed through exact rejection sampling with residual correction (following Leviathan and Chen). There is no second draft model taking up memory and no greedy shortcut, so the project says output stays correct at the model's native temperature. It is built on MLX, targets M1 or newer Macs on macOS 14+, and supports model families including Qwen 3.8 (27B and the 125B Flash Next mixture of experts), Prism ML's Ternary Bonsai 2 27B, Qwen 3.6, Qwen 3.5 and Gemma 4.
The project claims decode runs at around twice the speed of plain decoding — measured, by its own account, at 1.6x on a 16 GB M4 Mac mini and 2.24x on an M5 Max — and publishes the conditions and raw logs behind its numbers at mtplx.com/benchmarks. Models ship as prebuilt MTPLX packs on Hugging Face under the Youssofal account in Bare Speed, Optimized Speed and Optimized Quality variants, and the app and CLI recommend one that fits your Mac's memory (with FP16 builds picked automatically on M1 and M2). An auto-tune step measures each draft depth against ordinary autoregressive decoding on your own machine and only saves a depth that actually wins.
Around the engine sit a local server and tooling. mtplx start serves an OpenAI-compatible API (chat completions, responses, completions, models, optional embeddings and rerank) plus an Anthropic-compatible /v1/messages on 127.0.0.1:8000, so coding agents and clients such as OpenCode, Claude Code, Cline and Open WebUI can point at it. The app adds a live dashboard (tokens per second, acceptance rate by draft depth, cache state), native chat with file attachments and web search, fan control, a built-in AIME benchmark runner, and Forge, which converts a Hugging Face model to MLX, trains an MTP adapter, and verifies the result is faster and still exact before you publish it. mtplx inspect classifies a model's compatibility before anything runs, with no silent fallbacks. MTPLX is MLX-only; the README points Linux users to vLLM.
What it does
- Exact speculative decoding with the model's own MTP heads: one batched verify pass plus rejection sampling with residual correction, with no separate draft model
- Per-Mac auto-tune (mtplx tune --retune) that benchmarks each draft depth against plain autoregressive decoding and keeps only a depth that is faster
- Local OpenAI-compatible and Anthropic-compatible server on 127.0.0.1:8000, with streaming, tool calls, optional embeddings and reranking, and a disk-backed session cache that restores conversations across restarts
- Native Mac app with a live decode dashboard, streaming chat with file attachments and web search, one-click launch of coding agents, and a built-in AIME benchmark runner
- Forge: convert a Hugging Face model to MLX, train its MTP adapter, verify it is faster and exact on your hardware, and optionally publish it back to the Hub
- Hardware-aware model recommendations and a compatibility inspector that labels unverified or AR-only models instead of silently falling back
Getting started
MTPLX needs an Apple Silicon Mac (M1 or newer) on macOS 14+. The simplest path is the Mac app, downloaded as a DMG from mtplx.com, which checks your hardware, recommends and downloads a model that fits, sets up its own Python engine, puts mtplx on your PATH and tunes the decoding depth. The steps below use the CLI on its own, as the README shows.
Install the CLI and start it
Install with Homebrew (or python3 -m pip install mtplx), then run mtplx start to pick a model, mode and surface interactively and begin chatting.
brew install youssofal/mtplx/mtplx
mtplx start
Serve a specific model
Point the server at one of the MTPLX packs on Hugging Face. This example is the recommended coding pack on Flash Next for 128 GB Macs; smaller Macs get smaller packs such as Qwen3.8-27B-MTPLX-Optimized-Speed or Ternary-Bonsai-2-27B-MTPLX-Optimized-Speed.
mtplx serve --model Youssofal/Qwen3.8-Flash-Next-MTPLX-Optimized-SpeedCall the local API
The server speaks the OpenAI API (and the Anthropic /v1/messages API) on 127.0.0.1:8000, so any compatible client or coding agent can use it.
curl http://127.0.0.1:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"mtplx","messages":[{"role":"user","content":"hi"}],"stream":true}'Tune the draft depth for your Mac
Measure autoregressive decoding against each MTP depth on your own hardware; a depth is saved only if it beats the baseline.
mtplx tune --model <model-or-path> --retuneCheck health and compatibility
doctor reports install and integration health; inspect classifies a model before anything runs. Both work without MLX installed.
mtplx doctor
mtplx inspect <model>
mtplx modelsCommands and code are distilled from the project's own documentation — always check the official repo for the latest.
When to use it
- Reach for it when you want to run Qwen-family or Gemma models locally on an Apple Silicon Mac faster than plain decoding, without giving up sampling at the model's own temperature
- Reach for it when you need a private, local OpenAI- or Anthropic-compatible endpoint for coding agents such as OpenCode, Claude Code or Cline
- Reach for it when you want to build and verify your own MTP-enabled MLX model from a Hugging Face checkpoint and measure whether it is actually faster on your hardware
- Reach for it when one local daemon should serve chat plus embeddings and reranking for a RAG or agent-memory setup
How MTPLX compares
MTPLX alongside other open-source local runtimes tools AI/TLDR tracks, ranked by GitHub stars.
| Tool | Stars | What it does |
|---|---|---|
| Ollama | ★ 182k | A developer-friendly tool that downloads and runs local LLMs from the terminal with a built-in OpenAI-compatible API. |
| llama.cpp | ★ 130k | A C/C++ inference engine that runs LLMs in the GGUF format on CPUs, Apple Silicon, and GPUs with low memory use. |
| GPT4All | ★ 77.4k | GPT4All is a free desktop app and Python client that runs large language models locally on your own computer, with no API calls or GPU required. |
| LocalAI | ★ 49.3k | A self-hosted server that exposes an OpenAI-compatible API for running text, vision, voice, and image models on local hardware. |
| Jan | ★ 44.6k | An open-source desktop app that runs LLMs fully offline as a ChatGPT-style assistant on your own computer. |
| Colibrì | ★ 37.6k | A pure-C inference engine that keeps a Mixture-of-Experts model's dense trunk resident in RAM and streams its routed experts from disk, so 744B-2.8T models run on consumer hardware. |
| llmfit | ★ 37.2k | A Rust terminal tool that inspects your CPU, RAM, GPUs and VRAM and scores which open-weight models and quantizations will actually run well on that machine, with a TUI, CLI, REST API and local-runtime integrations. |
| MTPLX | ★ 2.4k | A native Mac app and CLI that runs local LLMs on Apple Silicon with the model's own multi-token prediction heads for exact speculative decoding |