CMU / Princeton · 2026-04-23 · major
Mamba-3 — Complex-Valued SSM with MIMO Updates Advances the Linear-Attention Frontier
Mamba-3 introduces trapezoidal discretization, complex-valued state updates, and MIMO formulation to advance state space models, gaining 1.8pp accuracy over Gated DeltaNet at 1.5B scale while halving state size vs Mamba-2.
Mamba-3 squeezes more from state space models with complex dynamics and MIMO — no extra parameters needed.
Key specs
| Accuracy gain over gated deltanet | 1.8pp |
|---|---|
| State size vs mamba2 | 50% |
| Scale | 1.5B |
What is it?
Mamba-3 is the third major iteration of the Mamba state space model architecture, published as a conference paper at ICLR 2026. It builds on Mamba-2 with three principled improvements derived from classical SSM theory. The work comes from Aakash Lahoti, Kevin Y. Li, Tri Dao, Albert Gu, and collaborators at CMU and Princeton.
How does it work?
The paper adds a trapezoidal discretization scheme (replacing Euler's method), a complex-valued state update rule that enables richer recurrence dynamics, and a Multi-Input Multi-Output (MIMO) formulation that increases arithmetic intensity during decoding without adding latency. The MIMO variant also theoretically connects complex-valued SSMs to Data-Dependent Rotary Embeddings (RoPE), bridging two popular design idioms.
Why does it matter?
Mamba-3 achieves comparable perplexity to Mamba-2 at half the state size, meaning the same capability with less memory. At 1.5B parameters it beats Gated DeltaNet by 1.8 percentage points on downstream tasks. As hybrid SSM-Transformer models become more common in production, improved SSM building blocks reduce the ceiling on what the non-attention layers can do.