vLLM · 2026-11-16 · major
vLLM v0.34.0 — a bigger KV cache by default, and Model Runner V1 is gone
In short
vLLM v0.34.0 brings 1,061 commits from 412 contributors. The KV cache is now sized from memory free after warmup, so most models get more cache. Model Runner V1 is removed, and Mistral Large 4 support arrives.
vLLM · 2026-11-16 · 2 sources checked

vLLM v0.34.0 gives most models a larger KV cache by default and makes Model Runner V2 the only GPU runner.
Quick facts
| Maker | vLLM project |
|---|---|
| Version | v0.34.0 |
| Released | November 16, 2026 |
| Contributors | 412 (120 new) across 1,061 commits |
| New models | 10, incl. Mistral Large 4, MiniMax-M3.1-Flash, LFM3 |
| Platform | PyTorch 2.14.0, CUDA 13.0, ROCm 10.0 only |
| Removed | Model Runner V1, QuTLASS, fp_quant, Marlin W4A8 |
What is it?
The extensible KV cache is now on by default in vLLM v0.34.0: on CUDA and ROCm the cache is sized from the memory that is actually free after CUDA graph capture and warmup, not from an estimate. The release notes give MiniMax-M3 on H200 as an example, with the cache growing from 433k to 672k tokens. The release also adds 10 new models, including Mistral Large 4 (1T total, 52B active, FP8 and NVFP4).
How does it work?
Model Runner V2 is now the only GPU runner, and setting `VLLM_USE_V2_MODEL_RUNNER=0` fails at startup. Suffix decoding runs on V2 without Arctic Inference, and a plugin API lets teams register their own speculators. For serving, new `/checkpoint/prepare`, `/checkpoint/status` and `/checkpoint/restore` endpoints work in both the Python and Rust frontends, alongside fault-tolerant DP+EP scale-down and NUMA binding on by default.
Why does it matter?
Operators get more cached tokens, and so more throughput, from the same GPUs without retuning. Upgrades need a check first: the default `--block-size` moves from 16 to 64, `gpu_memory_utilization` defaults to 1.0 while the extensible cache is on, and the release notes warn that per-token decode latency can rise on some models. The old behaviour comes back with `--gpu-memory-utilization 0.9` and `--no-enable-extensible-kv-cache`.
Who is it for?
inference and platform engineers
Frequently asked questions
- What breaks when upgrading to vLLM v0.34.0?
- vLLM v0.34.0 removes Model Runner V1, so `VLLM_USE_V2_MODEL_RUNNER=0` now fails at startup. The default `--block-size` changes from 16 to 64, ROCm 7.2 wheels and images are no longer published, QuTLASS, fp_quant and Marlin W4A8 are removed, and Keye-VL models are dropped. It also needs huggingface_hub 2.0.0 or newer.
- How do I turn off the extensible KV cache in vLLM v0.34.0?
- The extensible KV cache in vLLM v0.34.0 is on by default and sets `gpu_memory_utilization` to 1.0. To get the old sizing back, start the server with `--gpu-memory-utilization 0.9` and `--no-enable-extensible-kv-cache`. The release notes say this can matter for models where a larger cache raises per-token decode latency.
- Which new models can vLLM v0.34.0 serve?
- vLLM v0.34.0 adds 10 new models. They include Mistral Large 4 (1T total, 52B active, FP8 and NVFP4), MiniMax-M3.1-Flash, NVIDIA Nemotron-3.5-Transcribe, Zyphra ZAYA1, Baidu Unlimited-OCR-2, and Liquid AI's LFM3, LFM3-VL and LFM3-Audio. NVIDIA Parakeet TDT is supported through a plugin.
- What does vLLM v0.34.0 change for AMD and NVIDIA GPUs?
- On AMD, vLLM v0.34.0 ships ROCm 10.0 images and wheels only, adds MoRI EPv2, and makes TRITON_ATTN the default on RDNA3 and RDNA4. On NVIDIA, Step5 NVFP4 MoE on TRTLLM-Gen reaches up to 1.95x decode on 4xGB200, and GLM-5.3-Flash gets 1.6x output throughput, according to the release notes.
Try it
pip install -U vllm