Which new models does llama.cpp v0.4.0 support?

llama.cpp v0.4.0 adds initial architecture support for Qwen3.8-Flash-Next (named qwen4exp internally), NVIDIA Nemotron-3-Puzzle-75B-A9B, DSpark for Nemotron 3.5, and nanbeige4.2-3B. The release notes flag the Qwen3.8-Flash-Next path as initial support with optimization work still pending, so speed on that architecture should improve in later builds rather than being final today.

Does llama.cpp v0.4.0 break existing integrations?

llama.cpp v0.4.0 changes several public APIs: it adds llama_lazy_mode, adds max_buf_size to the quantize parameters, bumps the session and state versions for KV-cell token tracking, and makes mtmd_tokenize and mtmd_bitmap_init_lazy const-correct. Saved session files from earlier versions are affected by that version bump, so anything embedding libllama should re-check those call sites before upgrading.

How does v0.4.0 compare to v0.3.0?

v0.3.0 landed on 25 August 2026 and v0.4.0 on 4 September 2026, with more than 200 commits between them. The newer release is where lazy tensor reading, per-slot server context limits, video input options and the ggml 0.23.0 upgrade arrive. llama.cpp still publishes numbered build tags such as b10819 alongside these semantic-version releases.

Can llama.cpp be used commercially?

llama.cpp is published under the MIT license, which permits commercial use, modification and redistribution as long as the copyright notice is kept. The project sits at roughly 127,000 GitHub stars under the ggml-org organisation, and v0.4.0 is marked as a normal release rather than a pre-release, so it is the version the project points users at.

ggml-org · 2026-09-04 · major

llama.cpp v0.4.0 — Qwen3.8-Flash-Next support and lazy tensor loading

llama.cpp v0.4.0 adds initial support for Qwen3.8-Flash-Next and NVIDIA Nemotron-3-Puzzle-75B-A9B, plus lazy tensor reading that pulls weights from disk on demand. ggml moves to 0.23.0 with sparse flash attention and RDMA work.

The local LLM runtime picks up two new model architectures and learns to read weights from disk on demand.

Key specs

GitHub stars	127,174
Commits since v0.3.0	200+

Quick facts

Project	llama.cpp (ggml-org)
Version	v0.4.0
License	MIT
New model support	Qwen3.8-Flash-Next, Nemotron-3-Puzzle-75B-A9B, Nemotron 3.5 DSpark, nanbeige4.2-3B
ggml version	0.23.0
Previous release	v0.3.0 (25 August 2026)

What is it?

Lazy tensor reading is the headline change in llama.cpp v0.4.0: a new --lazy-mode flag makes the runtime read tensors only when they are needed instead of pulling the whole model in first. The release also adds four new model architectures, per-slot context limits in the server, and video input options for multimodal models.

How does it work?

Underneath, v0.4.0 ships ggml 0.23.0, which brings sparse flash attention for DeepSeek-V4, GLM and the new qwen4exp architecture, plus Apple RDMA as an RPC transport. Further memory work caps the quantizer's RAM use, streams rows in slabs while quantizing, and stops the load path from peaking. KV-cell token tracking and an n-gram history lookup speed up cache restore.

Why does it matter?

People running models on their own hardware feel these changes first. Support for Qwen3.8-Flash-Next means the open 125B mixture-of-experts model can run here soon after its release, and Nemotron-3-Puzzle-75B-A9B joins it. The memory work matters just as much: capping quantizer RAM and reading tensors on demand lower the hardware you need to prepare and start a large model.

Who is it for?

local LLM users, inference engineers

Frequently asked questions

Which new models does llama.cpp v0.4.0 support?: llama.cpp v0.4.0 adds initial architecture support for Qwen3.8-Flash-Next (named qwen4exp internally), NVIDIA Nemotron-3-Puzzle-75B-A9B, DSpark for Nemotron 3.5, and nanbeige4.2-3B. The release notes flag the Qwen3.8-Flash-Next path as initial support with optimization work still pending, so speed on that architecture should improve in later builds rather than being final today.
Does llama.cpp v0.4.0 break existing integrations?: llama.cpp v0.4.0 changes several public APIs: it adds llama_lazy_mode, adds max_buf_size to the quantize parameters, bumps the session and state versions for KV-cell token tracking, and makes mtmd_tokenize and mtmd_bitmap_init_lazy const-correct. Saved session files from earlier versions are affected by that version bump, so anything embedding libllama should re-check those call sites before upgrading.
How does v0.4.0 compare to v0.3.0?: v0.3.0 landed on 25 August 2026 and v0.4.0 on 4 September 2026, with more than 200 commits between them. The newer release is where lazy tensor reading, per-slot server context limits, video input options and the ggml 0.23.0 upgrade arrive. llama.cpp still publishes numbered build tags such as b10819 alongside these semantic-version releases.
Can llama.cpp be used commercially?: llama.cpp is published under the MIT license, which permits commercial use, modification and redistribution as long as the copyright notice is kept. The project sits at roughly 127,000 GitHub stars under the ggml-org organisation, and v0.4.0 is marked as a normal release rather than a pre-release, so it is the version the project points users at.

Try it

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp && git checkout v0.4.0