What breaks when upgrading to vLLM v0.28.0?

vLLM v0.28.0 lists four breaking changes. bitsandbytes support moves out of the main tree into a plugin, Transformers is bumped to 5.15.0, the deprecated calculate_kv_scales runtime KV scale calculation is removed, and override_attention_dtype is gone. Deployments pinning any of those need a config change before the upgrade.

Which models get faster in vLLM v0.28.0?

Kimi K3 is the main target of vLLM v0.28.0, with decode context parallelism, fused FlashKDA kernels, expert sharding and combined all-gathers. DeepSeek V4 is second: its sparse MLA path now works end to end for plain decode, MTP and speculative decoding. Mamba models also gain prefix caching by default.

Does vLLM v0.28.0 run on AMD GPUs?

vLLM v0.28.0 extends ROCm coverage. Kimi K3 runs on ROCm through the V2 model runner, DeepSeek V4 is enabled on gfx11 and gfx950, and AMD Quark NVFP4 quantization is supported for DeepSeek V4. AMD Quark and ROCm work sit alongside the CUDA path rather than replacing it.

What changed in the default settings?

Three defaults move in vLLM v0.28.0. max_num_batched_tokens rises from 8192 to 16384, prefix caching turns on by default for Mamba models, and the Blackwell CUDA graph capture default rises to 1024. Anyone tuning throughput against the old values should re-measure, since the batching change alone shifts memory use.

vLLM · 2026-08-26 · major

vLLM v0.28.0 — Kimi K3 gets a full-stack speed pass

vLLM v0.28.0 lands 584 commits from 270 contributors. The headline work is a Kimi K3 speed push — decode context parallelism, fused kernels and expert sharding that saves ~17 GiB per GPU — plus end-to-end sparse MLA for DeepSeek V4.

GitHub card for the vLLM inference and serving engine repository

The most-used open LLM serving engine ships a release built around making Kimi K3 and DeepSeek V4 cheaper to run.

Key specs

Commits	584
Contributors	270 (76 new)

Quick facts

Project	vLLM
Version	v0.28.0
Released	26 August 2026
License	Apache-2.0
GitHub stars	90,182
New default	max_num_batched_tokens 8192 to 16384
Breaking	bitsandbytes moves to an out-of-tree plugin

What is it?

Kimi K3 serving is the centre of vLLM v0.28.0. The release adds Decode Context Parallel support, fused FlashKDA decode and prefill kernels, SiTU activation for MegaMoE, and optional shared-expert sharding that frees roughly 17 GiB of memory on each GPU. Kimi K3 also now runs on ROCm through the V2 model runner. vLLM is the open inference and serving engine used to host open-weight models.

How does it work?

The speed comes from cutting communication and memory traffic rather than changing the models. Combined all-gathers give a 1.5–3x kernel-level speedup, GEMM-RS handles sequence parallelism, and an adaptive speculative token budget improves DSpark time-to-first-token by about 60%. DeepSeek V4's sparse MLA path now works end to end for plain decode, MTP and DSpark speculative decoding, with AMD Quark NVFP4 support and narrower eager CUDA graph regions.

Why does it matter?

Serving cost is the reason most teams cannot run a large open model, so a release that reclaims 17 GiB per GPU and widens AMD support changes what hardware a deployment needs. Tiered KV cache offloading now reaches disk, with out-of-tree secondary tier managers and tiering metrics. Two changes need attention before upgrading: bitsandbytes support moves to an out-of-tree plugin and Transformers jumps to 5.15.0.

Who is it for?

inference and platform engineers

Frequently asked questions

What breaks when upgrading to vLLM v0.28.0?: vLLM v0.28.0 lists four breaking changes. bitsandbytes support moves out of the main tree into a plugin, Transformers is bumped to 5.15.0, the deprecated calculate_kv_scales runtime KV scale calculation is removed, and override_attention_dtype is gone. Deployments pinning any of those need a config change before the upgrade.
Which models get faster in vLLM v0.28.0?: Kimi K3 is the main target of vLLM v0.28.0, with decode context parallelism, fused FlashKDA kernels, expert sharding and combined all-gathers. DeepSeek V4 is second: its sparse MLA path now works end to end for plain decode, MTP and speculative decoding. Mamba models also gain prefix caching by default.
Does vLLM v0.28.0 run on AMD GPUs?: vLLM v0.28.0 extends ROCm coverage. Kimi K3 runs on ROCm through the V2 model runner, DeepSeek V4 is enabled on gfx11 and gfx950, and AMD Quark NVFP4 quantization is supported for DeepSeek V4. AMD Quark and ROCm work sit alongside the CUDA path rather than replacing it.
What changed in the default settings?: Three defaults move in vLLM v0.28.0. max_num_batched_tokens rises from 8192 to 16384, prefix caching turns on by default for Mamba models, and the Blackwell CUDA graph capture default rises to 1024. Anyone tuning throughput against the old values should re-measure, since the batching change alone shifts memory use.

Try it

pip install -U vllm==0.28.0