Nanbeige LLM Lab · 2026-07-21 · major
Nanbeige4.2-3B — 3B looped-transformer open-weight agent hits 63.6 on SWE-Bench Verified
Nanbeige4.2-3B is a compact 3B open-weight model from BOSS Zhipin's Nanbeige LLM Lab. Its Looped Transformer reuses layers to raise capacity, and it scores 63.6 on SWE-Bench Verified and 87.4 on GPQA-Diamond.
A 3B open-weight agent whose Looped Transformer reuses layers to match models 3–4× its size.
Key specs
| Parameters | 4B total / 3B non-embedding |
|---|---|
| Context window | 256K |
| SWE-bench | 63.6 |
| GPQA | 87.4 |
Quick facts
| Maker | Nanbeige LLM Lab (BOSS Zhipin) |
|---|---|
| Parameters | 4B total, 3B non-embedding |
| Architecture | Looped Transformer |
| Context window | 262,144 tokens (256K) |
| Training data | 28T tokens (up from 23T) |
| License | Apache-2.0 |
| Availability | Open weights on Hugging Face; vLLM, SGLang, llama.cpp, Ollama recipes |
Benchmarks
| GPQA-Diamond | 87.4% | |
|---|---|---|
| HMMT-Feb-2026 | 82.8% | |
| LiveCodeBench-V6 | 72.5% | |
| SWE-Bench Pro | 46.9% | |
| Terminal-Bench 2.0 | 44.1% |
What is it?
Nanbeige4.2-3B is a compact open-weight language model built for agent work — tool use, code, reasoning — and released under Apache-2.0 on Hugging Face. It comes from Nanbeige LLM Lab, the AI research team inside Chinese recruiting platform BOSS Zhipin, and follows their Nanbeige4.1-3B from earlier this year.
How does it work?
The core trick is a Looped Transformer: hidden states pass bottom-to-top through the stack, then the model feeds them back in for another pass through the SAME layers. Layer reuse increases effective depth without adding parameters, giving a 3B backbone (with 4B total) capacity closer to 9–12B rivals. Training expanded from 23T to 28T tokens with heavier upsampling of math, code, STEM, and synthetic Q&A.
Why does it matter?
For local-first developers and agent builders, Nanbeige4.2-3B lands a real coding+reasoning model on a single consumer GPU. It reports 63.6 on SWE-Bench Verified — above Qwen3.5-9B (53.1) and Gemma4-12B (44.2) — plus 87.4 GPQA-Diamond, 46.9 SWE-Bench Pro, and 72.5 LiveCodeBench-V6, with day-one vLLM, SGLang, llama.cpp, and Ollama support and a 256K context.
Who is it for?
local-first developers, agent builders, small-model researchers
Frequently asked questions
- How does Nanbeige4.2-3B beat much bigger models?
- Nanbeige4.2-3B uses a Looped Transformer, feeding hidden states back into the same layers after a bottom-to-top pass. That reuse raises effective capacity without adding parameters, so a 3B backbone punches above its weight — 63.6 on SWE-Bench Verified against 53.1 for Qwen3.5-9B and 44.2 for Gemma4-12B.
- Is Nanbeige4.2-3B open source?
- Yes. Nanbeige LLM Lab (part of BOSS Zhipin) released Nanbeige4.2-3B under Apache-2.0 on Hugging Face, alongside a Nanbeige4.2-3B-Base checkpoint. Both weights and modeling code are downloadable, and the license permits commercial use with attribution.
- How do I run Nanbeige4.2-3B locally?
- Nanbeige ships day-one recipes for vLLM, SGLang, llama.cpp, and Ollama in the model card. With 3B non-embedding parameters and BF16 weights, the full model fits inside a single consumer GPU; a quantized GGUF build runs on CPUs and small VRAM setups.
- What is the context window?
- Nanbeige4.2-3B supports up to 262,144 tokens (256K) — long enough to read a full monorepo checkout or a long research thread in one shot. Tool-calling in both XML and JSON formats is built into the chat template, which the authors tuned for agentic workflows.
Try it
https://huggingface.co/Nanbeige/Nanbeige4.2-3B