How much RAM does llama-modchip need for DeepSeek-V4-Flash?

llama-modchip needs a 64 GB machine for the 284B DeepSeek-V4-Flash model. The README reports a peak resident size of 57.61 GiB at a 256K-token context, and about 57.6–58.9 GiB in general, against roughly 62.5 GiB usable on a 64 GB Linux system. The margin is tight, so close other large programs first. The 30B Nemotron model runs in 16–32 GB.

Can I load llama-modchip models in Ollama or LM Studio?

No. llama-modchip's Fractal-64 models use a custom tensor type, so unpatched llama.cpp, Ollama and LM Studio reject them with an "unknown tensor type 40" error. You run them through the project's ./modchip launcher or the prebuilt llama-server-modchip binary that ships next to the GGUF files on Hugging Face. That server is OpenAI-compatible, so existing clients can still connect to it.

Does llama-modchip work with a GPU too?

Yes, but llama-modchip does not need one. The main path is pure CPU, with kernels for AVX-512, AVX2, ARM NEON/SVE and Apple Metal. An optional hybrid mode keeps attention and SSM layers on a GPU while the routed experts stay in CPU RAM. Version 2.4.0 adds a Windows hybrid GPU+CPU pack and AMD Vulkan support for the Fractal-MXFP4 format.

How fast is llama-modchip compared with version 2.3?

On a Ryzen 9 9955HX with no GPU, llama-modchip 2.4.0 raised DeepSeek-V4-Flash prefill from about 25 tok/s to 43.16–44.36 tok/s, a 77.1% gain, and lossless decode from 5.91 tok/s to 9.06–9.95 tok/s. With top-4 expert routing, decode reaches 10.12–10.74 tok/s at a small quality cost. These are the project's own release-note numbers.

Synchopate · 2026-11-04 · major

llama-modchip 2.4 — 284B DeepSeek-V4-Flash on a 64 GB CPU, no GPU

In short

llama-modchip patches llama.cpp to run the 284B DeepSeek-V4-Flash MoE model in under 59 GiB of RAM at 256K context with no GPU. Version 2.4.0 raises CPU prefill by 77% to 44.36 tok/s.

Synchopate · 2026-11-04 · 3 sources checked

GitHub card for the synchopate/llama-modchip repository

Run a 284B open MoE model on a 64 GB desktop CPU, with a 256K context and no graphics card.

Key specs

Prefill (284 b, cpu)	44.36 tok/s
Peak ram at 256 k ctx	57.61 GiB

Quick facts

Maker	Synchopate (open-source project)
Version	v2.4.0 (2026-11-04)
License	Apache-2.0
Main model	DeepSeek-V4-Flash, 284.33B total / 13.17B active
Model size	56.56 GiB at 1.71 bits per weight
Hardware	64 GB RAM, no GPU (30B models: 16–32 GB)
Decode speed (284B)	9.1–14.7 tok/s, project's figures

What is it?

llama-modchip is a set of llama.cpp kernel patches, 2-bit quant formats and launcher tools that fits DeepSeek-V4-Flash, a 284B Mixture-of-Experts model, into the RAM of a 64 GB machine. Version 2.4.0, released on 2026-11-04, is the build that reached 460 points on Hacker News this weekend. It also runs smaller MoE models such as Nemotron-3.5-30B-A3B in 16–32 GB.

How does it work?

A custom "Fractal-64" format shrinks the DeepSeek checkpoint from 76.87 GiB to 56.56 GiB, so the weights and a 256K KV cache stay in memory without swap. New AVX-512 kernels keep quantized weights in CPU registers across up to 8 tokens during prompt reading, and an MQA8 FlashAttention path halved attention time from 415.1 ms to 208.5 ms. A tiny n-gram drafter does speculative decoding, and KV snapshots in /dev/shm restore a session in about 13 ms.

Why does it matter?

Running a frontier-size open model usually means renting GPUs or buying several. This project shows a single consumer CPU with 64 GB of RAM can serve it privately, at reading speed, for agent tools like Claude Code. The catch: the models use custom tensor types, so stock llama.cpp, Ollama and LM Studio cannot load them, and all speed figures are the project's own.

Who is it for?

local-LLM users with 64 GB machines and no GPU

Frequently asked questions

How much RAM does llama-modchip need for DeepSeek-V4-Flash?: llama-modchip needs a 64 GB machine for the 284B DeepSeek-V4-Flash model. The README reports a peak resident size of 57.61 GiB at a 256K-token context, and about 57.6–58.9 GiB in general, against roughly 62.5 GiB usable on a 64 GB Linux system. The margin is tight, so close other large programs first. The 30B Nemotron model runs in 16–32 GB.
Can I load llama-modchip models in Ollama or LM Studio?: No. llama-modchip's Fractal-64 models use a custom tensor type, so unpatched llama.cpp, Ollama and LM Studio reject them with an "unknown tensor type 40" error. You run them through the project's ./modchip launcher or the prebuilt llama-server-modchip binary that ships next to the GGUF files on Hugging Face. That server is OpenAI-compatible, so existing clients can still connect to it.
Does llama-modchip work with a GPU too?: Yes, but llama-modchip does not need one. The main path is pure CPU, with kernels for AVX-512, AVX2, ARM NEON/SVE and Apple Metal. An optional hybrid mode keeps attention and SSM layers on a GPU while the routed experts stay in CPU RAM. Version 2.4.0 adds a Windows hybrid GPU+CPU pack and AMD Vulkan support for the Fractal-MXFP4 format.
How fast is llama-modchip compared with version 2.3?: On a Ryzen 9 9955HX with no GPU, llama-modchip 2.4.0 raised DeepSeek-V4-Flash prefill from about 25 tok/s to 43.16–44.36 tok/s, a 77.1% gain, and lossless decode from 5.91 tok/s to 9.06–9.95 tok/s. With top-4 expert routing, decode reaches 10.12–10.74 tok/s at a small quality cost. These are the project's own release-note numbers.

Try it

git clone https://github.com/synchopate/llama-modchip.git && cd llama-modchip && ./modchip chat