Code Llama 70B

// Overview

Code Llama 70B is the largest model in Meta's Code Llama family, released on January 29, 2024. It is fine-tuned from Llama 2 and specialized for generating, explaining and completing code. The 70B tier was added after the original 7B, 13B and 34B models and was trained on roughly 1 trillion tokens of code and code-related data — about double the 500B tokens used for the smaller sizes.

Meta shipped three 70B variants: Code Llama 70B (the foundation model), Code Llama - Python 70B (further tuned on Python), and Code Llama - Instruct 70B (tuned to follow natural-language instructions and to be safer and more helpful in chat). All three support large inputs — the models are trained on 16k-token sequences and show improvements on contexts up to 100k tokens — and they handle popular languages including Python, C++, Java, PHP, TypeScript, C# and Bash.

Code Llama 70B is released under the Llama 2 Community License, which permits both research and commercial use. The weights are openly available, so the model is most commonly self-hosted or run through community tooling such as Ollama (codellama:70b) and Hugging Face rather than a first-party paid API. It has since been superseded by Meta's general-purpose Llama 3 family for many coding tasks, but it remains a notable open-weight code model.

// Benchmarks

1. HumanEval (pass@1) — Code Llama - Instruct 70B67.8%
2. MBPP (pass@1) — Code Llama - Instruct 70B62.2%
3. HumanEval (pass@1) — Code Llama - Python 70B57.3%
4. MBPP (pass@1) — Code Llama - Python 70B65.6%
5. HumanEval (pass@1) — Code Llama 70B (base)53%
6. MBPP (pass@1) — Code Llama 70B (base)62.4%

Scores on a 0–100 scale (25-point gridlines); higher is better. Each benchmark links to its published source.

// Pricing

Meta does not sell Code Llama 70B through a first-party API. The weights are released for free under the Llama 2 Community License and can be self-hosted (e.g. via Ollama or Hugging Face); third-party hosted inference prices vary by provider.

Pricing source ↗

// Strengths

• Largest open-weight model in the Code Llama line, with strong HumanEval/MBPP results among open models at release
• Three variants let you pick the right fit: foundation, Python-specialized, or instruction-tuned for chat-style coding
• Permissive Llama 2 Community License allows commercial use
• Handles large inputs — trained on 16k-token sequences with improvements up to 100k tokens
• Open weights mean it can be fully self-hosted (Ollama, Hugging Face, llama.cpp) with no per-token API cost

// Best for

• Local or on-prem code generation and autocompletion where data can't leave your infrastructure
• Code explanation, review and debugging assistance via the Instruct variant
• Python-heavy workloads using the Code Llama - Python 70B variant
• Fine-tuning a code model on a private codebase from an openly licensed base
• Research and benchmarking of open code models

// How to access

// Code Llama — every version

The full lineage of the Code Llama line, newest first. Every version has its own page — click any to compare specs, benchmarks and pricing.