Can SGLang v0.5.17 run Kimi K3 on AMD GPUs?

Yes. SGLang verified its Kimi K3 day-0 path on both NVIDIA GB300 and AMD MI35x hardware. The model ships as a native MXFP4 checkpoint, and SGLang serves it with DCP, DSpark speculative decoding, chunked-prefill pipeline parallelism with tensor-parallel decode, KDA-aware prefix caching, LoRA on the quantized weights, and OpenAI-compatible reasoning and tool-call endpoints.

What is DWDP in SGLang and when should I turn it on?

DWDP is a mixture-of-experts prefill strategy in SGLang v0.5.17 that prefetches peer expert weights over NVLink peer-to-peer and computes every expert locally, removing the expert-parallel all-to-all token dispatch. Enable it with the --dwdp-size flag. The authors mark it early-development, so treat it as experimental rather than a default for production serving.

Does SGLang v0.5.17 make restarting a large model faster?

SGLang v0.5.17 adds a weight-cache daemon that holds model weights per GPU so a restarting engine recovers from cache instead of reloading from storage. The release notes put current large-model restarts at three to six-plus minutes, and about 6.5 minutes for Qwen3-235B in FP8 on four GPUs, because weights reload and CUDA graphs recapture.

How does SGLang help agent workloads specifically?

SGLang v0.5.17 adds a session-reference-aware unified radix cache for agentic and reinforcement-learning rollout workloads. Requests can carry a stable session_id so cache eviction knows which prefixes a live session still needs, instead of evicting purely by policy. Release the references with /close_session and opt in with --enable-session-radix-cache.

SGLang · 2026-08-08 · major

SGLang v0.5.17 — day-0 serving for Kimi K3 and MiniMax H3

SGLang v0.5.17 serves Kimi K3, the 2.8T-parameter open model, and MiniMax H3 video generation from day one. It also starts moving the request front-end from Python to Rust. 582 pull requests from 194 contributors.

SGLang GitHub repository card for the high-performance LLM serving framework

SGLang v0.5.17 serves two frontier open models on release day and starts replacing its Python front-end with Rust.

Key specs

Mo e prefill speedup	1.92x over DEP4
Prefill throughput	506K tok/s at saturation

Quick facts

Version	v0.5.17
Released	8 August 2026
Scope	582 pull requests from 194 contributors
New day-0 models	Kimi K3 and MiniMax H3
Also added	EmbeddingGemma, LFM2.5, MiniMax-M3-NVFP4
Verified hardware	NVIDIA GB300, B200, H100, RTX 5090; AMD MI35x
License	Apache-2.0

What is it?

Day-0 serving for Kimi K3 lands in SGLang v0.5.17 — Moonshot's 2.8T-parameter LatentMoE model with a 1M-token context, 896 experts and a MoonViT3d vision tower, running on NVIDIA GB300 and AMD MI35x. The same release serves MiniMax H3, which returns a video and a matching stereo audio track from a single request, plus the EmbeddingGemma and LFM2.5 embedding models.

How does it work?

A new Rust serving layer handles everything from network ingress and tokenization to an OpenAI-compatible API, taking that work off the Python path. On the throughput side, a prefill strategy called DWDP prefetches peer expert weights over NVLink and computes all experts locally, dropping the all-to-all token dispatch — 1.92x faster than DEP4 on four B200s with gpt-oss-120b, and 506K versus 329K tokens per second at saturation.

Why does it matter?

Teams self-hosting open weights can run Kimi K3 and MiniMax H3 the week they land rather than waiting for engine support, which is the usual gap between a Chinese lab shipping weights and anyone serving them at scale. Faster mixture-of-experts prefill and a session-aware cache also change what a GPU-hour buys, and SGLang is one of the engines most open deployments sit on.

Who is it for?

inference and platform engineers self-hosting open models

Frequently asked questions

Can SGLang v0.5.17 run Kimi K3 on AMD GPUs?: Yes. SGLang verified its Kimi K3 day-0 path on both NVIDIA GB300 and AMD MI35x hardware. The model ships as a native MXFP4 checkpoint, and SGLang serves it with DCP, DSpark speculative decoding, chunked-prefill pipeline parallelism with tensor-parallel decode, KDA-aware prefix caching, LoRA on the quantized weights, and OpenAI-compatible reasoning and tool-call endpoints.
What is DWDP in SGLang and when should I turn it on?: DWDP is a mixture-of-experts prefill strategy in SGLang v0.5.17 that prefetches peer expert weights over NVLink peer-to-peer and computes every expert locally, removing the expert-parallel all-to-all token dispatch. Enable it with the --dwdp-size flag. The authors mark it early-development, so treat it as experimental rather than a default for production serving.
Does SGLang v0.5.17 make restarting a large model faster?: SGLang v0.5.17 adds a weight-cache daemon that holds model weights per GPU so a restarting engine recovers from cache instead of reloading from storage. The release notes put current large-model restarts at three to six-plus minutes, and about 6.5 minutes for Qwen3-235B in FP8 on four GPUs, because weights reload and CUDA graphs recapture.
How does SGLang help agent workloads specifically?: SGLang v0.5.17 adds a session-reference-aware unified radix cache for agentic and reinforcement-learning rollout workloads. Requests can carry a stable session_id so cache eviction knows which prefixes a live session still needs, instead of evicting purely by policy. Release the references with /close_session and opt in with --enable-session-radix-cache.

Try it

https://docs.sglang.io/cookbook/autoregressive/Moonshotai/Kimi-K3