# A 10-Year-Old Xeon Is All You Need — Christina Sørensen Runs Gemma 4 26B-A4B MoE on a 2016 E5-2620 v4 With No GPU at ~12 Tokens/sec

> Modern frontier MoE inference, on a decade-old server CPU, at human reading speed.

- Org: point.free
- Date: 2026-06-01
- Category: showcase, tutorial
- Source: https://point.free/blog/gemma-4-on-a-2016-xeon/
- Canonical page: https://ai-tldr.dev/releases/cafkafk-gemma-4-on-2016-xeon/

## In short

Christina Sørensen (cafkafk) gets Gemma 4's 26B Mixture-of-Experts checkpoint generating at reading speed on a decade-old Xeon E5-2620 v4 with 128GB DDR3 RAM and no GPU, leaning on ik_llama.cpp, MTP speculative decoding, CPU Flash Attention, MLA compression, and runtime expert-routing tuning.

## What it is

A long-form engineering write-up from Christina Sørensen — NixOS Steering Committee member and eza maintainer — describing what it actually took to run Google's Gemma 4 26B-A4B Mixture-of-Experts model on a single 2016-era Intel Xeon E5-2620 v4 with 128GB of DDR3, no GPU, and a 262K context window. The blog post hit #3 on Hacker News with 672 points and 272 comments on June 1.

## How it works

Sørensen uses ik_llama.cpp (a fork tuned for CPU inference) plus a stack of optimizations: MTP-drafter-based speculative decoding to reduce memory-bandwidth pressure on the verifier, a CPU-side Flash Attention implementation that avoids materializing N×N matrices, Multi-Head Latent Attention to compress the KV cache, runtime weight repacking aligned to L2/L3 cache lines, and explicit expert-routing tuning to keep the MoE from thrashing the cache. Total footprint lands at ~82GB (25GB weights + 56GB KV at 262K context) and the system generates at ~11.94 tokens/sec.

## Why it matters

It's a real data point for the 'frontier MoE inference can run on commodity CPUs' thesis. Most local-LLM commentary assumes consumer GPUs are the floor; this post documents the exact knobs and trade-offs that turn a recycled 2016 server into a usable Gemma 4 endpoint, and the comment thread has practitioners trading numbers from their own rigs.

## Who it's for

Local-LLM tinkerers, CPU-inference engineers, and anyone running models on recycled hardware.

## Try it

Read the post on point.free/blog/gemma-4-on-a-2016-xeon

## Links

- [Blog Post](https://point.free/blog/gemma-4-on-a-2016-xeon/)
- [MTP Prequel](https://point.free/blog/gemma-4-mtp/)
- [HN Discussion](https://news.ycombinator.com/item?id=48353348)
- [Author on GitHub](https://github.com/cafkafk)