Kimi Linear (48B-A3B)

// Overview

Kimi Linear (48B-A3B) is an open-weight large language model from Moonshot AI (the team behind Kimi). It is the flagship release of the Kimi Linear line, published on October 31, 2025 alongside the technical report "Kimi Linear: An Expressive, Efficient Attention Architecture." Moonshot ships it in two checkpoints on Hugging Face — Kimi-Linear-48B-A3B-Base and Kimi-Linear-48B-A3B-Instruct — both under the permissive MIT license.

The model is a Mixture-of-Experts design with 48B total parameters but only about 3B active per token (hence the "A3B" name), trained on 5.7T tokens. Its headline feature is the attention stack: instead of using full attention everywhere, Kimi Linear interleaves three layers of Kimi Delta Attention (KDA) — a linear-attention mechanism that refines Gated DeltaNet with finer gating — for every one layer of Multi-Head Latent Attention (MLA). This 3:1 hybrid is meant to be a drop-in replacement for full attention.

The payoff is long-context efficiency. Moonshot reports the architecture cuts KV-cache memory by up to 75% and delivers up to 6x faster decoding throughput at a 1M-token context, while matching or beating full attention on quality. That makes Kimi Linear (48B-A3B) most interesting to people who want to self-host a long-context model cheaply rather than to chase the absolute top of the reasoning leaderboards.

// Benchmarks

1. MMLU-Pro (4k context)51%
2. RULER (128k context)84.3%

Scores on a 0–100 scale (25-point gridlines); higher is better. Each benchmark links to its published source.

// Strengths

• Native 1M-token context window with hardware-efficient KDA kernels
• Up to 75% smaller KV cache versus standard full-attention models
• Up to 6x faster decoding throughput at 1M-token context (6.3x faster time-per-output-token vs MLA per the paper)
• Sparse MoE: only ~3B of 48B parameters active per token, keeping inference cost low
• Fully open weights under the permissive MIT license (Base + Instruct checkpoints)
• Open-sourced KDA kernels with vLLM and Flash Linear Attention (FLA) integration

// Best for

• Self-hosted long-context applications (large-document QA, multi-file code analysis) where KV-cache memory is the bottleneck
• High-throughput batch and agent workloads that need fast decoding at long context
• Cost-sensitive deployments that want an MoE model with low active-parameter count
• Research and experimentation on linear / hybrid attention architectures, using the open KDA kernels
• Fine-tuning the Base checkpoint for domain-specific tasks under a permissive MIT license

// How to access