JetBrains · 2026-06-01 · notable
JetBrains Mellum2 — 12B Mixture-of-Experts With 2.5B Active Parameters Opens Up Under Apache 2.0, Ships Base, Instruct, and Thinking Variants Trained on ~10.6T Tokens
JetBrains open-sourced Mellum2, a 12B Mixture-of-Experts model with 64 experts and 8 routed per token (~2.5B active), trained from scratch on roughly 10.6T tokens for routing, RAG, and sub-agent workloads inside coding pipelines.
A fast, open 12B MoE designed for routing, RAG, and sub-agents inside coding workflows.
Key specs
| License | Apache 2.0 |
|---|---|
| Parameters | 12B |
| Active params | 2.5B |
| Context window | 128K |
| Experts | 64 (8 active) |
| Training tokens | ~10.6T |
| Layers | 28 |
What is it?
Mellum2 is JetBrains' second open-weight model, a 12B Mixture-of-Experts language model with 64 experts and 8 routed per token, giving an active compute footprint of about 2.5B parameters. It succeeds the original 4B Mellum from 2025 and broadens the scope from raw code completion to general code-plus-text reasoning, with base, instruct, instruct-SFT, thinking, and thinking-SFT checkpoints all on Hugging Face.
How does it work?
The model was trained from scratch on roughly 10.6 trillion tokens through a three-phase curriculum that progressively shifts toward code and math, with a 128K context window obtained via layer-selective YaRN. Architecturally it runs 28 layers at hidden size 2304, with 32 query heads and 4 KV heads under grouped-query attention in bfloat16. JetBrains positions it as a 'focal' model: small enough per token to embed inside a request-heavy agent — routing, classification, RAG post-processing — yet competitive with 4B–14B dense baselines on benchmarks like LiveCodeBench v6, BFCL v4, and GSM-Plus.
Why does it matter?
Most open MoE releases target frontier-scale workloads on H100s. Mellum2 inverts that: it is tuned for the dense, frequent rest of the agent stack — deciding which tool to call, which document to fetch, or whether a step is done. Apache 2.0 weights make it deployable inside private IDE installs and CI agents, which fits JetBrains' AI Assistant strategy and gives teams an alternative to leaning on a frontier API for cheap orchestration calls.
Who is it for?
agent builders and IDE-tooling teams that need a fast routing or RAG model
Try it
huggingface.co/JetBrains/Mellum2-12B-A2.5B-Instruct