SGLang · 2026-09-18 · major
SGLang v0.5.20 — CUDA 12 wheels retired, radix cache covers every model
SGLang v0.5.20 lands 713 pull requests from 237 contributors and stops publishing CUDA 12 wheels and images. Its unified radix tree now caches branching points, lifting the token hit rate on DeepSeek-V4-Flash from 43.8% to 60.8%.
The September SGLang release trades CUDA 12 support for a smarter prefix cache, faster RL rollouts and nine more models.
Quick facts
| Version | v0.5.20 |
|---|---|
| Released | September 18, 2026 |
| Scope | 713 PRs from 237 contributors |
| Biggest break | No more CUDA 12 wheels or images |
| Required dependency | FlashInfer 0.6.18 |
| Default ROCm lane | ROCm 10 |
| License | Apache-2.0 |
What is it?
Caching is where v0.5.20 changes most. SGLang's unified radix tree now records branching points, so requests that share a long system prompt reuse far more of it: on DeepSeek-V4-Flash the token hit rate goes from 43.8% to 60.8% and mean time-to-first-token falls from 1.57 s to 1.07 s. SGLang is an open-source serving framework for large language, multimodal and diffusion models.
How does it work?
Reinforcement-learning rollouts get a dedicated path: sampling masks with overlap scheduling raise decode throughput on Qwen3-8B by 17% at batch 1 and 52% at batch 64 over the previous implementation, with capacity set by the new --sampling-mask-max-tokens flag. A new HRRN scheduler policy cuts mean time-to-first-token by 69% against first-come-first-served on a GLM-5.2 production trace. On Blackwell, TRT-LLM attention kernels give DeepSeek-V4 about 1.2x faster prefill and 1.45x faster decode than FlashMLA at kernel level on a B200.
Why does it matter?
The upgrade is not free: v0.5.19 was the last release with CUDA 12 wheels and images, prefill context parallelism v1 is gone, and /v1/responses stops keeping results in memory unless the server starts with --enable-response-store. Teams on AMD get the largest single win — GLM-5.2 at TP4 on four MI355X cards now loads in 40.4 s instead of 505.7 s — and ROCm 10 becomes the default lane. Nine more architectures are servable, including GLM-5.3-Flash, Qwen3.8-Flash-Next, K2 Horizon, Nanbeige4.2, Tencent's Hy4-Preview and, for the first time, Mamba 1 and Mamba 2 state-space models.
Who is it for?
inference and RL-infrastructure teams
Frequently asked questions
- What breaks when upgrading to SGLang v0.5.20?
- SGLang v0.5.20 publishes no CUDA 12 wheels or images at all, so anyone still on CUDA 12 must pin v0.5.19. Prefill context parallelism v1 and its legacy CLI options are removed, /v1/responses needs --enable-response-store to keep results, the sampling-mask environment variable is rejected at start-up in favour of --sampling-mask-max-tokens, and ServerArgs moved to msgspec so dataclass introspection no longer works.
- Which new models does SGLang v0.5.20 serve?
- SGLang v0.5.20 adds GLM-5.3-Flash, Tencent's Hy4-Preview, Qwen3.8-Flash-Next, K2 Horizon and Nanbeige4.2 on the autoregressive side, plus the SenseNova-U1.5-8B-MoT, FastH3 and VDN-H3 diffusion models. It also gains support for Mamba 1 and Mamba 2 state-space models, widening the framework beyond transformer architectures.
- How much does SGLang v0.5.20 help on AMD hardware?
- AMD sees the release's largest jump: loading GLM-5.2 at TP4 across four MI355X cards drops from 505.7 s to 40.4 s. MiniMax-M3 output throughput rises between 6.4% and 20.6% at 80K input on eight MI350X cards, and ROCm 10 is now the default lane, with ROCm 7.0 CI, nightly images and kernel wheels retired.
- Can you predict SGLang serving latency before deploying?
- SGLang v0.5.20 ships a simulator that estimates time-to-first-token within roughly 6% of measured serving data on most traces. That gives capacity planners a way to size a deployment without standing up the full cluster first, which pairs well with the new HRRN scheduler policy for tuning queueing behaviour under load.
Try it
pip install "sglang==0.5.20"