Google DeepMind · 2026-06-05 · major
Google Ships Gemma 4 QAT Checkpoints — New Mobile Quantization Format Brings Gemma 4 E2B Under 1GB of RAM, Q4_0 Weights Land for E2B, E4B, 12B, and the 26B MoE on Hugging Face
Google releases quantization-aware-trained Gemma 4 weights in the popular Q4_0 format and a new mobile format that pushes E2B text-only under 1GB of RAM, with day-one llama.cpp, Ollama, MLX, vLLM, and SGLang support.
Gemma 4 shrinks to under 1GB on phones via a custom 2-bit mobile quantization format trained with QAT.
Key specs
| E2 b mobile memory | < 1 GB |
|---|---|
| Q4 0 sizes shipped | E2B, E4B, 12B, 26B MoE |
| Mobile decode bits | 2 |
What is it?
A set of post-training checkpoints for Gemma 4 — Google's open-weight multimodal LLM family — that were compressed using Quantization-Aware Training (QAT) instead of standard post-training quantization. Two formats ship: the popular Q4_0 (4-bit) used by llama.cpp, and a new Google mobile format that uses 2-bit decoding layers, optimized KV caches, and static activations.
How does it work?
QAT simulates the integer rounding of low-bit quantization during fine-tuning so the model is trained to be robust to the quality drop that post-training quantization (PTQ) normally introduces. Google reports QAT yields higher overall quality than its PTQ baselines at matching bit-widths. The mobile format pushes generation layers to 2 bits while keeping activations static, and pairs the encoder-free Gemma 4 architecture with a slimmer text-only path that drops the audio and vision encoders.
Why does it matter?
The E2B text-only variant fits in under 1GB of RAM, putting a real multimodal-trained Google LLM inside the budget of mid-range phones and low-end laptops with no GPU. Day-one support across llama.cpp, Ollama, LM Studio, vLLM, SGLang, MLX, LiteRT-LM, Transformers.js, Hugging Face Transformers, and Unsloth means the new weights drop straight into the existing local-LLM tooling stack.
Who is it for?
Local-LLM hobbyists, on-device app builders, mobile ML engineers.
Try it
huggingface.co/google/gemma-4-E2B-it-qat-mobile-transformers