Overview
Gemma 4 12B is the unified, encoder-free entry in Google DeepMind's Gemma 4 open-weight family, released on June 3, 2026 under the Apache 2.0 license. It is a roughly 12-billion-parameter dense transformer (11.95B) that accepts text, images, audio waveforms and video as input and returns text, without the separate vision or audio encoders used by the rest of the Gemma 4 line.
Where the April 2026 Gemma 4 models (E2B, E4B, 26B MoE and 31B dense) pair a language model with dedicated vision and audio stacks, the 12B variant projects raw image patches and audio waveforms directly into the LLM's embedding space through lightweight linear layers. Google reports the model delivers performance nearing the 26B Mixture-of-Experts model on standard benchmarks at less than half the total memory footprint, and fits comfortably on machines with 16GB of VRAM or unified memory.
The model carries a 256K-token context window, was trained on more than 140 languages, and is available with day-one support across Hugging Face Transformers, llama.cpp, MLX, SGLang, vLLM, Unsloth, LM Studio and Ollama. Weights ship from Hugging Face and Kaggle; production deployments are available through Google Cloud's Model Garden, Cloud Run and GKE.
| Released | 2026-06-03 |
|---|---|
| License | Apache 2.0 |
| Weights | Open weights |
| Parameters | 11.95B |
| Context | 256K |
| Architecture | Dense transformer with hybrid local/global attention (48 layers, 262K vocab). Encoder-free "unified" multimodal stack: images, audio waveforms and video are projected directly into the LLM's embedding space through lightweight linear layers — no separate vision or audio encoder. |
| Knowledge cutoff | January 2025 |
| Modalities | Text, Vision, Audio, Video |
| Status | Generally available |
Benchmarks
- MMLU Pro77.2%
- GPQA Diamond78.8%
- AIME 2026 (no tools)77.5%
- LiveCodeBench v672%
- MATH-Vision79.7%
- MMMU Pro69.1%
- BigBench Extra Hard53%
- MRCR v2 (8 needle, 128K)43.4%
Scores on a 0–100 scale (25-point gridlines); higher is better. Each benchmark links to its published source.
Strengths
- Encoder-free unified multimodal architecture — handles text, image, audio and video without separate encoders
- 256K-token context for long-document and long-context tasks
- Strong reasoning and math for its size: 77.5% on AIME 2026 (no tools), 78.8% on GPQA Diamond, 79.7% on MATH-Vision
- Strong code generation for an open ~12B model: 72.0% on LiveCodeBench v6 and a 1659 Codeforces ELO
- Runs locally on consumer hardware with 16GB of VRAM or unified memory
- Apache 2.0 license permits commercial use, fine-tuning and self-hosting without a custom Gemma license
- Day-one support across Transformers, llama.cpp, MLX, SGLang, vLLM, Unsloth, LM Studio and Ollama
Best for
- Local and on-device multimodal applications that need image, audio or video understanding alongside text
- Self-hosted assistants and agents on workstations or single-GPU servers
- Long-context document analysis and retrieval-augmented generation with the 256K window
- Fine-tuning a permissively-licensed multimodal base for domain-specific tasks
- Multilingual applications spanning 140+ languages
- Code and math assistants where an open ~12B model is the right size/cost trade-off
How to access
| Provider | Model ID |
|---|---|
| Google Cloud Vertex AI Model Garden ↗ | gemma-4-12b |
| Hugging Face ↗ | google/gemma-4-12B-it |
| Ollama ↗ | gemma-4-12b |
Gemma (open weights) — every version
The full lineage of the Gemma (open weights) line, newest first. Every version has its own page — click any to compare specs, benchmarks and pricing.
FAQ
What is Gemma 4 12B?
Gemma 4 12B is the unified, encoder-free entry in Google's Gemma 4 open-weight family, released on June 3, 2026 under the Apache 2.0 license. It is a roughly 12-billion-parameter dense transformer that accepts text, images, audio waveforms and video and projects them directly into the LLM's embedding space rather than going through separate vision or audio encoders.
How does Gemma 4 12B differ from the rest of Gemma 4?
The April 2026 Gemma 4 release covered four sizes — E2B, E4B, 26B MoE and 31B dense — each pairing a language model with separate vision and audio encoders. Gemma 4 12B drops the encoders entirely and projects raw image patches and audio waveforms directly into the LLM via lightweight linear layers. Google reports it reaches performance nearing the 26B MoE on standard benchmarks at less than half the memory footprint.
What hardware does Gemma 4 12B need?
Gemma 4 12B runs locally on machines with about 16GB of VRAM or unified memory, putting it within reach of consumer GPUs and Apple Silicon laptops. It ships with day-one support across Hugging Face Transformers, llama.cpp, MLX, SGLang, vLLM, Unsloth, LM Studio and Ollama, plus production deployment on Google Cloud Model Garden, Cloud Run and GKE.
What can Gemma 4 12B do?
It handles text generation, multilingual conversation in 140+ languages, code generation, math and reasoning, plus native image, audio and video understanding without separate encoders. The 256K-token context window supports long-document analysis and retrieval-augmented generation, and the Apache 2.0 license permits commercial use, fine-tuning and self-hosting.