Google Research / Google DeepMind / NYU · 2026-04-23 · major
TurboQuant — Google's ICLR 2026 KV Cache Compression: 6x Memory, 8x Speed, Zero Accuracy Loss
TurboQuant uses PolarQuant rotation + 1-bit QJL residual correction to quantize the LLM KV cache to 3 bits with zero accuracy loss, achieving 6x memory reduction and up to 8x attention speedup on H100 GPUs, presented at ICLR 2026.
TurboQuant compresses the KV cache to 3 bits with zero accuracy loss — 6x less memory and 8x faster attention on H100s.
Key specs
| Kv cache memory reduction | 6x |
|---|---|
| Attention speedup h100 | 8x |
| Quantization bits | 3 |
| Accuracy loss | zero |
What is it?
TurboQuant is a KV cache quantization algorithm from Google Research, Google DeepMind, and NYU presented at ICLR 2026. It addresses one of the main memory bottlenecks in long-context LLM inference: the growing key-value cache that stores all previous tokens' attention representations.
How does it work?
The method uses a two-stage approach. PolarQuant randomly rotates each key and value vector to redistribute variance evenly, then converts to polar coordinates for quantization — eliminating outliers that cause conventional quantization to fail. A 1-bit Quantized Johnson-Lindenstrauss (QJL) residual layer corrects remaining errors without requiring any retraining.
Why does it matter?
Long-context models with million-token windows are increasingly standard, but the KV cache grows linearly with context length and can exceed GPU memory. TurboQuant's training-free 3-bit compression with zero accuracy degradation below 3.5 bits per channel makes it directly applicable to deployed models — reducing inference costs and enabling longer effective contexts on existing hardware.