oMLX · 2026-08-18 · notable
oMLX 0.6.2 — the Mac LLM server now tunes its own ANE/GPU split
oMLX 0.6.2 adds a built-in tuner that benchmarks Apple Neural Engine and GPU prefill splits on your own Mac, plus dedicated GPU kernels for M5 chips. The Apache-2.0 inference server has 19,733 GitHub stars.
Instead of shipping one ANE/GPU ratio for every Mac, oMLX now measures the best split on the machine in front of you.
Key specs
| License | Apache-2.0 |
|---|---|
| GitHub stars | 19,733 |
What is it?
A built-in ANE/GPU split tuner is the main change in oMLX 0.6.2, released on 18 August 2026. oMLX is an Apache-2.0 LLM inference server for Apple Silicon with continuous batching and SSD caching, managed from the macOS menu bar. It keeps the KV cache in a hot in-memory tier and a cold SSD tier, so earlier context stays reusable across requests.
How does it work?
The tuner benchmarks a GPU-only baseline, several MLP-only splits and combined MLP/GDN splits on the current Mac, from either the macOS app or the web dashboard. Candidate settings stay temporary until the user applies them, and the tuner recommends GPU-only when the best split is under 1% faster. New NAX QMM kernels handle the GPU half of hybrid prefill on M5-family Macs for quantized Qwen3.5, Qwen3.6 and Qwen3.8 layers, and oMLX falls back to the existing Metal path when NAX is unavailable.
Why does it matter?
Split ratios tuned on one Mac ran slower on another, which is why oMLX had disabled the Neural Engine on M5 machines outright; measuring on the actual device replaces that guesswork, and the blanket M5 block is gone. Community testing on M5 reported about a 5% improvement, with the best tuner trial approaching 1,000 prompt tokens per second. The release also fixes a crash when Lightning MTP ran with TurboQuant KV enabled, and restores exact FP32 SSD-cache snapshots as the default so a restored cache no longer changes greedy outputs.
Who is it for?
people running local models on Apple Silicon
Try it
brew install jundot/omlx/omlx