# llama-modchip 2.4 — 284B DeepSeek-V4-Flash on a 64 GB CPU, no GPU

> Run a 284B open MoE model on a 64 GB desktop CPU, with a 256K context and no graphics card.

- Org: Synchopate
- Date: 2026-11-04
- Category: repo, tool
- Source: https://github.com/synchopate/llama-modchip
- Canonical page: https://ai-tldr.dev/releases/synchopate-llama-modchip-2-4-0/

## In short

llama-modchip patches llama.cpp to run the 284B DeepSeek-V4-Flash MoE model in under 59 GiB of RAM at 256K context with no GPU. Version 2.4.0 raises CPU prefill by 77% to 44.36 tok/s.

## What it is

llama-modchip is a set of llama.cpp kernel patches, 2-bit quant formats and launcher tools that fits DeepSeek-V4-Flash, a 284B Mixture-of-Experts model, into the RAM of a 64 GB machine. Version 2.4.0, released on 2026-11-04, is the build that reached 460 points on Hacker News this weekend. It also runs smaller MoE models such as Nemotron-3.5-30B-A3B in 16–32 GB.

## How it works

A custom "Fractal-64" format shrinks the DeepSeek checkpoint from 76.87 GiB to 56.56 GiB, so the weights and a 256K KV cache stay in memory without swap. New AVX-512 kernels keep quantized weights in CPU registers across up to 8 tokens during prompt reading, and an MQA8 FlashAttention path halved attention time from 415.1 ms to 208.5 ms. A tiny n-gram drafter does speculative decoding, and KV snapshots in /dev/shm restore a session in about 13 ms.

## Why it matters

Running a frontier-size open model usually means renting GPUs or buying several. This project shows a single consumer CPU with 64 GB of RAM can serve it privately, at reading speed, for agent tools like Claude Code. The catch: the models use custom tensor types, so stock llama.cpp, Ollama and LM Studio cannot load them, and all speed figures are the project's own.

## Who it's for

local-LLM users with 64 GB machines and no GPU

## Try it

git clone https://github.com/synchopate/llama-modchip.git && cd llama-modchip && ./modchip chat

## Quick facts

- Maker: Synchopate (open-source project)
- Version: v2.4.0 (2026-11-04)
- License: Apache-2.0
- Main model: DeepSeek-V4-Flash, 284.33B total / 13.17B active
- Model size: 56.56 GiB at 1.71 bits per weight
- Hardware: 64 GB RAM, no GPU (30B models: 16–32 GB)
- Decode speed (284B): 9.1–14.7 tok/s, project's figures

## FAQ

### How much RAM does llama-modchip need for DeepSeek-V4-Flash?

llama-modchip needs a 64 GB machine for the 284B DeepSeek-V4-Flash model. The README reports a peak resident size of 57.61 GiB at a 256K-token context, and about 57.6–58.9 GiB in general, against roughly 62.5 GiB usable on a 64 GB Linux system. The margin is tight, so close other large programs first. The 30B Nemotron model runs in 16–32 GB.

### Can I load llama-modchip models in Ollama or LM Studio?

No. llama-modchip's Fractal-64 models use a custom tensor type, so unpatched llama.cpp, Ollama and LM Studio reject them with an "unknown tensor type 40" error. You run them through the project's ./modchip launcher or the prebuilt llama-server-modchip binary that ships next to the GGUF files on Hugging Face. That server is OpenAI-compatible, so existing clients can still connect to it.

### Does llama-modchip work with a GPU too?

Yes, but llama-modchip does not need one. The main path is pure CPU, with kernels for AVX-512, AVX2, ARM NEON/SVE and Apple Metal. An optional hybrid mode keeps attention and SSM layers on a GPU while the routed experts stay in CPU RAM. Version 2.4.0 adds a Windows hybrid GPU+CPU pack and AMD Vulkan support for the Fractal-MXFP4 format.

### How fast is llama-modchip compared with version 2.3?

On a Ryzen 9 9955HX with no GPU, llama-modchip 2.4.0 raised DeepSeek-V4-Flash prefill from about 25 tok/s to 43.16–44.36 tok/s, a 77.1% gain, and lossless decode from 5.91 tok/s to 9.06–9.95 tok/s. With top-4 expert routing, decode reaches 10.12–10.74 tok/s at a small quality cost. These are the project's own release-note numbers.

## Links

- [GitHub Repo](https://github.com/synchopate/llama-modchip)
- [v2.4.0 Release Notes](https://github.com/synchopate/llama-modchip/releases/tag/v2.4.0)
- [284B Model on HF](https://huggingface.co/Synchopate/DeepSeek-V4-Flash-0731-Fractal-64GB-CPU-GGUF)
- [HN Discussion](https://news.ycombinator.com/item?id=50411961)