Gemma 1

// Overview

Gemma 1 is the first generation of Google's open-weights large language model family, announced February 21, 2024 and developed by Google DeepMind. It was released in two sizes — Gemma 2B and Gemma 7B — each available as a pre-trained base model and an instruction-tuned (IT) chat variant. Google positioned Gemma as a lightweight, openly distributed counterpart to its closed Gemini models, sharing Gemini's tokenizer, 256,128-token vocabulary, and core architectural research while shrinking the weights down to a size that runs on a single consumer GPU, a workstation, or a laptop.

Architecturally, Gemma 1 is a decoder-only transformer with an 8,192-token context window, RoPE embeddings, GeGLU activations, and RMSNorm. The 7B model uses standard multi-head attention; the 2B model uses multi-query attention and was distilled from the larger model. The 2B variant has about 2.5 billion total parameters and the 7B about 8.5 billion (the headline names refer to non-embedding parameter counts). Google trained the 2B model on roughly 3 trillion tokens and the 7B on roughly 6 trillion tokens of mostly-English web text, code, and math, using TPUv5e with JAX. The instruction-tuned variants were aligned with supervised fine-tuning and reinforcement learning from human feedback.

At launch Gemma was distributed under the custom Gemma Terms of Use — a source-available license that permits responsible commercial use and redistribution but carries a Prohibited Use Policy and flow-down obligations, rather than the Apache 2.0 license Google later adopted for newer Gemma generations. Google shipped toolchain support across JAX, PyTorch, and Keras 3.0, plus Hugging Face, NVIDIA NeMo, and TensorRT-LLM, and made the weights available through Kaggle, Hugging Face, and Vertex AI. An April 5, 2024 maintenance update introduced Gemma 1.1, with improved instruction-tuned checkpoints. Gemma 1 has since been superseded by Gemma 2, Gemma 3, and Gemma 4.

// Benchmarks

1. MMLU (5-shot)64.3%
2. HellaSwag (0-shot)81.2%
3. HumanEval (pass@1)32.3%
4. GSM8K (maj@1)46.4%
5. MATH (4-shot)24.3%
6. ARC-c53.2%
7. PIQA (0-shot)81.2%
8. BoolQ (0-shot)83.2%

Scores on a 0–100 scale (25-point gridlines); higher is better. Each benchmark links to its published source.

// Strengths

• First Google open-weights LLM — downloadable weights you can self-host, fine-tune, and deploy commercially under the Gemma Terms of Use
• Small enough to run on a single consumer GPU, workstation, or laptop (2B and 7B sizes)
• Built on Gemini research: shared tokenizer, 256k vocabulary, and architecture, distilled into open weights
• Strong benchmark results for its size class at launch, beating similarly-sized open models on 11 of 18 text tasks per the technical report
• Broad framework support out of the box: JAX, PyTorch, Keras 3.0, Hugging Face, NeMo, TensorRT-LLM
• Both pre-trained base and instruction-tuned chat variants for each size

// Best for

• Running a capable open LLM locally for privacy-sensitive or offline workloads
• Fine-tuning a small open base model on proprietary data with LoRA/QLoRA
• Text generation, summarization, and chatbot prototyping on modest hardware
• On-device and edge deployments where a 2B model fits the compute budget
• Research and reproducibility work that needs open weights rather than an API
• Cost-controlled self-hosted inference as an alternative to closed hosted APIs

// How to access

// Gemma (open weights) — every version

The full lineage of the Gemma (open weights) line, newest first. Every version has its own page — click any to compare specs, benchmarks and pricing.