Is the faster startup path in SGLang v0.5.18 on by default?

No. The overlapped checkpoint staging in SGLang v0.5.18 is opt-in — pass --startup-weight-load-mode overlap when you launch the server. Against a serial load that already prefetches, the overlap mode saves 8.6 to 11.7 percent on Qwen3-32B on an H100. Against the plain default it is 2.38 times faster.

What breaks when I upgrade to SGLang v0.5.18?

SGLang v0.5.18 moves the Triton, FlashInfer, Inductor, DeepGEMM and CUDA driver caches into one directory controlled by SGLANG_CACHE_DIR. The old caches are no longer read, so the first launch after upgrading recompiles kernels once and takes longer than usual. Later launches reuse the new directory normally.

Which GPUs benefit from the FlashInfer allreduce change?

The pure-allreduce path in SGLang v0.5.18 reuses the FlashInfer MNNVL workspace instead of falling back to NCCL, and the measured gain is on Blackwell — DeepSeek-V4-Flash decode at TP4 improves up to 6.9 percent at small batch sizes. SGLang turns it on automatically for DeepSeek V3, V3.2 and V4; elsewhere use --enable-flashinfer-pure-allreduce.

How does SGLang v0.5.18 differ from v0.5.17?

Version 0.5.17 was about day-0 coverage for two frontier open models, Kimi K3 and MiniMax H3, across 582 pull requests. SGLang v0.5.18 is larger at 710 pull requests and aims at overhead instead — startup time, decode-step cost and one cache directory — while adding seven mostly multimodal and diffusion model families.

SGLang · 2026-08-22 · major

SGLang v0.5.18 — cold starts get 2.38x faster, seven model families land

SGLang v0.5.18 stages model weights from storage while CUDA graphs capture, so a Qwen3-32B server on an H100 starts in 35.6 seconds instead of 84.8. Seven new model families get serving support. 710 pull requests from 212 contributors.

SGLang GitHub repository card for the open-source LLM and multimodal serving framework

SGLang v0.5.18 overlaps weight loading with CUDA graph capture, cutting a large-model cold start from 85 seconds to 36.

Key specs

Qwen3 32 b startup	35.6s vs 84.8s (2.38x)
Lmhead decode step	320us to 169us

Quick facts

Version	v0.5.18
Released	22 August 2026
Scope	710 pull requests from 212 contributors
New model families	Muse Glimmer, Intern-S2-Mobius, SANA-Video, LingBot-Video-MoE, LTX-2.5, Cosmos3 Edge & Distilled, LongCat-Image
Core dependencies	torch 2.13.0, triton 3.7.1, flashinfer 0.6.17, sgl-kernel 0.4.6.post1
Breaking change	All compiled-kernel caches move under SGLANG_CACHE_DIR
License	Apache-2.0

What is it?

Overlapped checkpoint staging is the headline change in SGLang v0.5.18. The server used to read model weights off storage and then capture CUDA graphs; now the two run at the same time. On an H100 serving Qwen3-32B that turns an 84.8-second cold start into 35.6 seconds. The release also adds serving recipes for seven new model families, most of them multimodal or diffusion — Muse Glimmer, Intern-S2-Mobius, SANA-Video, LingBot-Video-MoE, LTX-2.5, Cosmos3 Edge and Distilled, and LongCat-Image.

How does it work?

Three separate overheads get trimmed. Checkpoint pages stage from storage in parallel with graph capture, behind the --startup-weight-load-mode overlap flag. The tensor-parallel LMHead replaces an allgather plus scatter with a single all-to-all under pure data-parallel attention, which drops that step from 320 to 169 microseconds on a DeepSeek-V4-Pro B200 decode and moves time-per-output-token from 36.97 to 35.67 milliseconds. Non-fused allreduce sites reuse the FlashInfer MNNVL workspace rather than falling back to NCCL.

Why does it matter?

Restart time is a real operational cost for anyone running large open models: autoscaling, rolling upgrades and crash recovery all pay it, and 85 seconds per replica adds up across a fleet. Halving it makes a serving pool cheaper to move around. The microsecond-level decode savings matter less per request but compound at high throughput, and SGLang is one of the two engines most self-hosted open-weight deployments sit on.

Who is it for?

inference and platform engineers self-hosting open models

Frequently asked questions

Is the faster startup path in SGLang v0.5.18 on by default?: No. The overlapped checkpoint staging in SGLang v0.5.18 is opt-in — pass --startup-weight-load-mode overlap when you launch the server. Against a serial load that already prefetches, the overlap mode saves 8.6 to 11.7 percent on Qwen3-32B on an H100. Against the plain default it is 2.38 times faster.
What breaks when I upgrade to SGLang v0.5.18?: SGLang v0.5.18 moves the Triton, FlashInfer, Inductor, DeepGEMM and CUDA driver caches into one directory controlled by SGLANG_CACHE_DIR. The old caches are no longer read, so the first launch after upgrading recompiles kernels once and takes longer than usual. Later launches reuse the new directory normally.
Which GPUs benefit from the FlashInfer allreduce change?: The pure-allreduce path in SGLang v0.5.18 reuses the FlashInfer MNNVL workspace instead of falling back to NCCL, and the measured gain is on Blackwell — DeepSeek-V4-Flash decode at TP4 improves up to 6.9 percent at small batch sizes. SGLang turns it on automatically for DeepSeek V3, V3.2 and V4; elsewhere use --enable-flashinfer-pure-allreduce.
How does SGLang v0.5.18 differ from v0.5.17?: Version 0.5.17 was about day-0 coverage for two frontier open models, Kimi K3 and MiniMax H3, across 582 pull requests. SGLang v0.5.18 is larger at 710 pull requests and aims at overhead instead — startup time, decode-step cost and one cache directory — while adding seven mostly multimodal and diffusion model families.

Try it

uv pip install --prerelease=allow sglang