Kyle Howells · 2026-06-12 · notable
Kyle Howells: How to set up a local coding agent on macOS — 72 tokens/sec on Gemma 4
Step-by-step recipe for running a local coding agent on Apple Silicon with llama.cpp, Gemma 4 26B-A4B, and MTP speculative decoding. Lifts throughput from 58 to 72 tokens/sec. 352 points on HN.
A working recipe for a local coding agent on a Mac, with measured throughput numbers and an OpenAI-compatible endpoint.
Key specs
| Baseline | 58.2 tok/sec |
|---|---|
| With mtp draft | 72.2 tok/sec |
| Qwen 3.6 35 b a3 b | 55 tok/sec |
| Hn points | 352 |
What is it?
Kyle Howells walks through running an agentic coding setup entirely on a Mac, with no API keys. The stack is llama.cpp with Metal acceleration, Gemma 4 26B-A4B (Q4) as the main model, and a multimodal projector for image input. A coding-agent framework called Pi hits the model through an OpenAI-compatible local endpoint.
How does it work?
He configures llama.cpp to serve Gemma 4 with an MTP (Multi-Token Prediction) draft model for speculative decoding, which lifts throughput from 58.2 to 72.2 tokens per second — about 24%. He also benchmarks Qwen 3.6 35B-A3B as an alternative at 55 tok/sec, slower but reportedly better at coding tasks. The post includes the exact CLI flags, model paths, and how to wire Pi to the local endpoint.
Why does it matter?
Local coding agents have been slow and fiddly, with most setup posts skipping the speed numbers. This one is end-to-end on a Mac, lists actual tokens-per-second, and ships speculative decoding as a first-class step — useful for anyone who wants Copilot-class help without the cloud round-trip or API bill.
Who is it for?
Mac developers, local-LLM tinkerers, privacy-sensitive teams
Try it
https://ikyle.me/blog/2026/how-to-setup-a-local-coding-agent-on-macos