NVIDIA · 2026-04-07 · notable
TriAttention — trigonometric KV cache compression
A new attention scheme that compresses the KV cache by exploiting Q/K vector concentration in pre-RoPE space. Targets long-context inference memory bottlenecks.
NVIDIA's new attention technique compresses the KV cache by exploiting a geometric property of Q/K vectors before RoPE is applied.
What is it?
TriAttention is a new variant of attention designed to make long-context LLM inference cheaper. It targets the KV cache — the chunk of GPU memory that stores per-token keys and values during generation, which dominates memory cost as context length grows past 100k tokens.
How does it work?
The authors observe that Q and K vectors concentrate in specific angular regions in the pre-RoPE (rotary positional embedding) space. TriAttention uses this concentration to estimate the importance of each cached key cheaply, without computing the full attention matrix. Less-important keys get evicted or compressed; the rest stay in cache. The trigonometric framing comes from working in the angular RoPE space.
Why does it matter?
Long-context models all hit the same wall: the KV cache eats GPU memory faster than the model's parameters do. KV compression that does not degrade quality is the bottleneck on cheap 1M-context serving. If TriAttention's quality holds at scale it pushes that wall back significantly.
Who is it for?
ML systems researchers, anyone building inference infrastructure for long-context models.