Mistral 7B

// Overview

Mistral 7B is Mistral AI's first model, released on 2023-09-27 under the Apache 2.0 license with fully downloadable weights. It is a 7.3-billion-parameter dense transformer that, in the launch paper (arXiv 2310.06825), outperformed Meta's Llama 2 13B on every benchmark the authors tested and matched or beat the much larger Llama 1 34B on reasoning, mathematics, and code. That result — a 7B model beating a 13B one — is what put Mistral AI on the map.

Architecturally, Mistral 7B introduced two efficiency tricks that later became standard across the field: grouped-query attention (GQA), which shrinks the KV cache and speeds up decoding, and sliding-window attention (SWA) with a 4,096-token window, which lets each layer attend locally while information still propagates across stacked layers. The original v0.1 release shipped with an 8K context; the later v0.2 and v0.3 weight updates raised the effective context to 32K, and v0.3 extended the vocabulary to 32,768 tokens and added function-calling support.

Mistral 7B is now a legacy model. On Mistral's hosted API (la Plateforme) the open-mistral-7b endpoint was deprecated on 2024-11-30 and retired on 2025-03-30, with Ministral 8B named as its successor. The open weights remain freely available on Hugging Face and run locally through Ollama, llama.cpp, vLLM, and LM Studio, so the model lives on as a lightweight, edge-friendly base for fine-tuning even though Mistral no longer serves it directly.

// Benchmarks

1. MMLU (5-shot)60.1%
2. HellaSwag (0-shot)81.3%
3. WinoGrande (0-shot)75.3%
4. PIQA (0-shot)83%
5. ARC-Challenge (0-shot)55.5%
6. TriviaQA (5-shot)69.9%
7. HumanEval (0-shot)30.5%
8. MBPP (3-shot)47.5%
9. GSM8K (8-shot, maj@8)52.2%
10. MATH (4-shot, maj@4)13.1%

Scores on a 0–100 scale (25-point gridlines); higher is better. Each benchmark links to its published source.

// Strengths

• Small enough to run on a single consumer GPU (or quantized on a laptop) while outscoring the 2x-larger Llama 2 13B across the launch-paper benchmark suite
• Permissive Apache 2.0 license with no usage restrictions — free to fine-tune, redistribute, and deploy commercially
• Efficient inference from grouped-query attention plus sliding-window attention, lowering memory and latency versus a vanilla 7B transformer
• Strong math and code for its size: 52.2% GSM8K and 30.5% HumanEval in the paper, approaching Code-Llama 7B on code without sacrificing general performance
• Huge ecosystem support (Hugging Face, Ollama, vLLM, llama.cpp, LM Studio) and a vast library of community fine-tunes built on it

// Best for

• A lightweight, locally-deployable base model for fine-tuning on domain-specific tasks
• On-device and edge assistants where a small, fast, permissively-licensed model is required
• Cost-sensitive chat, summarization, and classification workloads that don't need a frontier model
• A teaching and research baseline for studying GQA and sliding-window attention
• Self-hosted inference for privacy-sensitive deployments that cannot send data to a hosted API

// How to access

// Mistral 7B / Nemo (open dense) — every version

The full lineage of the Mistral 7B / Nemo (open dense) line, newest first. Every version has its own page — click any to compare specs, benchmarks and pricing.