How much faster is llama.cpp with the Cua Metal shim?

The Cua post publishes three measured pairs on an M1 Ultra. TinyLlama 1.1B goes from 12.63 to 206.60 tokens per second when generating, a 16.36x gain. Gemma 4 12B goes from 3.41 to 49.67 tokens per second, 14.54x. Muse Glimmer 30B in a 64 GiB guest goes from 2.38 to 21.08 tokens per second, 8.87x.

Does the Cua shim need a patched hypervisor or a modified macOS guest?

No. The Cua Metal capability shim is process-scoped: it is injected into one guest process with DYLD_INSERT_LIBRARIES and changes nothing in the underlying virtualization infrastructure. You do have to stop the VM once and set a macOS preference for unrestricted device features, and Lume's documentation includes LaunchAgent templates and rollback steps.

Is the Metal capability shim safe to rely on?

Cua calls the Metal capability shim experimental and version-sensitive, because it depends on private Metal implementation details. It only affects processes you inject it into, and only the capability profiles Cua has tested. Other Apple chips, other guest macOS releases, and other Metal APIs each need separate validation before you trust the result.

Why is llama.cpp so slow in a macOS VM to begin with?

Inside a macOS guest, Metal reports a lower capability profile than the same Apple Silicon hardware reports on the host, including a smaller 32 KB threadgroup memory limit. llama.cpp reads those answers and falls back to slower kernels. The Cua shim raises the reported Apple family to 9 and the threadgroup limit to 64 KB so the fast kernel paths become eligible.

Cua · 2026-08-11 · major

Metal Capability Shim — llama.cpp runs up to 16x faster inside macOS VMs

Cua's Metal capability shim lets llama.cpp reach the fast Metal GPU kernels inside a macOS virtual machine on Apple Silicon. On an M1 Ultra, token generation runs 8.9x to 16.4x faster and prompt processing 7.2x to 11.1x faster.

Cua repository card for the macOS VM GPU passthrough write-up

A process-scoped Metal shim unlocks the fast GPU kernels for llama.cpp inside macOS VMs on Apple Silicon.

Key specs

Token generation	8.9–16.4x faster
Prompt processing	7.2–11.1x faster

Quick facts

Maker	Cua (trycua)
License	MIT, in the Cua repository
Code path	libs/lume/metal-capability-shim
Tested on	M1 Ultra, 48-core GPU, macOS 26.6.1
Status	Experimental research release
What it changes	Metal Apple family up to 9, threadgroup memory 32 KB to 64 KB

What is it?

The Metal capability shim is a small compatibility layer that changes selected capability answers Metal gives to one process inside a macOS virtual machine. Francesco Bonacci and Johnny Franks published it on 11 August 2026 as a research release in the Cua repository, the 21,000-star project for running computer-use agents across operating systems. Before it, running a local model in a Cua VM meant single-digit tokens per second.

How does it work?

Injection happens through DYLD_INSERT_LIBRARIES, so only the process you target sees the changed answers. When llama.cpp asks Metal what the GPU supports, the shim reports a higher Apple family level and a larger threadgroup memory budget, and llama.cpp selects its optimized kernels instead of the conservative fallback. Environment variables such as LUME_METAL_APPLE_FAMILY_MAX control the reported profile.

Why does it matter?

VMs are how teams sandbox coding agents and computer-use agents on Macs, and until now that sandbox made local inference impractical: a 30B model generated about 2.4 tokens per second. At 21 tokens per second the same setup becomes usable, so an agent can run a local model inside the isolated environment instead of calling out to a hosted API.

Who is it for?

Mac developers running agents or local LLMs in VMs

Frequently asked questions

How much faster is llama.cpp with the Cua Metal shim?: The Cua post publishes three measured pairs on an M1 Ultra. TinyLlama 1.1B goes from 12.63 to 206.60 tokens per second when generating, a 16.36x gain. Gemma 4 12B goes from 3.41 to 49.67 tokens per second, 14.54x. Muse Glimmer 30B in a 64 GiB guest goes from 2.38 to 21.08 tokens per second, 8.87x.
Does the Cua shim need a patched hypervisor or a modified macOS guest?: No. The Cua Metal capability shim is process-scoped: it is injected into one guest process with DYLD_INSERT_LIBRARIES and changes nothing in the underlying virtualization infrastructure. You do have to stop the VM once and set a macOS preference for unrestricted device features, and Lume's documentation includes LaunchAgent templates and rollback steps.
Is the Metal capability shim safe to rely on?: Cua calls the Metal capability shim experimental and version-sensitive, because it depends on private Metal implementation details. It only affects processes you inject it into, and only the capability profiles Cua has tested. Other Apple chips, other guest macOS releases, and other Metal APIs each need separate validation before you trust the result.
Why is llama.cpp so slow in a macOS VM to begin with?: Inside a macOS guest, Metal reports a lower capability profile than the same Apple Silicon hardware reports on the host, including a smaller 32 KB threadgroup memory limit. llama.cpp reads those answers and falls back to slower kernels. The Cua shim raises the reported Apple family to 9 and the threadgroup limit to 64 KB so the fast kernel paths become eligible.

Try it

./Scripts/build.sh in libs/lume/metal-capability-shim