DeepSeek · 2026-09-17 · major
DeepSeek V4.1 Flash report — the KV cache drops to 890 bytes a token
DeepSeek published the technical report behind DeepSeek V4.1 Flash. It shows how cross-layer KV cache reuse and 4-bit KV storage cut the always-in-memory cache to 890 bytes per token, about a quarter of DeepSeek V4 Flash.
The paper behind DeepSeek V4.1 Flash: how a 552B model keeps only 890 bytes of cache per token.
Key specs
| Global kv cache | 890 B / token |
|---|
Quick facts
| Maker | DeepSeek |
|---|---|
| Paper | arXiv 2609.19969, 17 Sep 2026 |
| Backbone | 552B multimodal MoE |
| Active params | 8B prefill / 16B decode |
| Context window | 1M tokens |
| Pretraining data | 45T multimodal tokens |
| Weights | Open on Hugging Face |
What is it?
Two new memory tricks are the heart of the DeepSeek V4.1 Flash report: cross-layer KV cache reuse inside Compressed Sparse Attention 2 (CSA2), and storing the KV cache in FP4. Together they shrink the global cache that must stay in HBM to 890 bytes per token, roughly a quarter of what DeepSeek V4 Flash needed. The model itself is a 552B multimodal Mixture-of-Experts with a one-million-token context.
How does it work?
A Causal Encoder-Decoder layout lets DeepSeek V4.1 Flash activate only 8B parameters while reading the prompt and 16B while writing the answer, so the expensive prefill stage costs less. CSA2 reuses compressed key-value state across layers instead of storing a fresh copy per layer, and FP4 storage shrinks each entry to four bits. A separate deployment trick, SWA Bounded Replay, cuts the persistent cache that lives on SSD or host memory to about an eighth of the previous model's.
Why does it matter?
Long-running agents send huge inputs, and the KV cache is what fills up expensive GPU memory and saturates data transfer. By publishing the numbers and the method, DeepSeek gives inference teams a concrete recipe for serving million-token contexts on less hardware, and gives researchers a baseline to compare their own cache-compression work against.
Who is it for?
inference engineers and efficiency researchers
Frequently asked questions
- How much smaller is the cache than DeepSeek V4 Flash?
- DeepSeek V4.1 Flash holds 890 bytes per token in its global KV cache, which the report puts at roughly one quarter of DeepSeek V4 Flash. The persistent cache that sits on SSD or in host memory falls further, to about one eighth, using a deployment technique the paper calls SWA Bounded Replay.
- Does the smaller cache cost accuracy?
- No. The DeepSeek V4.1 Flash report states the model delivers substantially better performance than the baseline despite the much smaller KV cache footprint, across text and multimodal agentic tasks. The gains come from streamlining the DeepSeek V4 architecture and adding efficiency extensions alongside the compression work.
- Can I download the weights?
- Yes. DeepSeek publishes model checkpoints for DeepSeek V4.1 Flash on Hugging Face, and the paper links them directly from the abstract. The weights were posted before this report, so the paper documents a model people can already run rather than announcing a new one.
- What is CSA2?
- Compressed Sparse Attention 2 is the attention scheme DeepSeek V4.1 Flash uses, extended in this report with cross-layer KV cache reuse so layers share compressed key-value state instead of each keeping its own copy. Combined with FP4 storage for those entries, it is the main source of the 890-bytes-per-token figure.
Try it
https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash