Apple Machine Learning Research · 2026-10-15 · notable
Apple runs Muse Spark Open on four Mac Studios — 36 tokens/s with MLX and RDMA
Apple's ML Research post shows how to run large open models on Mac Studio with MLX, then split them across machines with RDMA over Thunderbolt. Four M5 Ultra Mac Studios run Meta's 816B Muse Spark Open at 36.73 tokens/s.
A hands-on Apple guide to running near-trillion-parameter open models on a small cluster of Mac Studios.
Key specs
| Muse spark open, 4x m5 ultra | 36.73 tok/s |
|---|---|
| Qwen 3.8 27 b, 4x m5 ultra | 50.89 tok/s |
What is it?
Apple Machine Learning Research published a guide to local and distributed LLM inference with MLX on Mac Studio. It benchmarks Qwen 3.8 27B and Mistral Medium 3.5 on one machine, then Meta's Muse Spark Open (816B total, 42B active) across two and four M5 Ultra Mac Studios. In NVFP4 the Muse model needs about 493GB for its weights alone, so it cannot fit on one 512GB machine.
How does it work?
MLX v0.30.1 or later adds the jaccl backend, which uses the RDMA-over-Thunderbolt driver in macOS 26.2. The mlx.launch tool starts a job across a hostfile of machines and splits the model by layers (pipeline parallelism) or by width (tensor parallelism). Each pair of machines is linked by two Thunderbolt cables to double bandwidth, and mlx_lm.server exposes an OpenAI-compatible endpoint.
Why does it matter?
Frontier-size open weights no longer need a data-center GPU server to run privately. Tensor parallelism across four machines more than doubles single-node decode speed for the smaller models, and the post shows the setup driving OpenCode as a local coding agent.
Who is it for?
developers running large open models locally
Try it
mlx_lm.benchmark --model Qwen/Qwen3.8-27B -p 8192 -g 128 -b 1