Is LiteLLM v1.98.0 a stable release or a preview?

LiteLLM v1.98.0 is published as a stable release on GitHub, not a pre-release, while the v1.99.0 line was still shipping release candidates and dev builds at the same time. The Docker image ghcr.io/berriai/litellm:v1.98.0 is signed with cosign, and the release notes give the verification command against a pinned commit hash.

How do I turn on provisioned-throughput billing in LiteLLM?

Flat-cost attribution in LiteLLM v1.98.0 is gated behind an opt-in environment variable, so it stays off until you switch it on. After that you enter the PTU inputs on the model deployment in the model form, and daily rollups write the per-model flat cost by active hour so the Usage page can show it.

What is a shadow eval in LiteLLM's auto-router?

A shadow eval lets a LiteLLM operator test a routing change before adopting it. The auto-router samples live /v1/messages and /v1/responses traffic and has a blind pairwise judge compare the candidate routing against the one in use, in either direction. A direction picker and reverse-mode display sit in the shadow evals tab.

Which new model pricing does LiteLLM v1.98.0 add?

LiteLLM v1.98.0 adds price entries for NVIDIA Nemotron 3.5 Lightning on both OpenRouter and DeepInfra. LiteLLM keeps a model-price map that drives its cost tracking, so a model needs an entry there before spend on it can be attributed.

Does LiteLLM v1.98.0 change anything for MCP servers?

Yes. LiteLLM v1.98.0 scopes gateway session bearer tokens to the RFC 8707 resource, so an MCP session token is tied to the resource it was issued for. The release also exposes client HTTP headers to logging callbacks and hooks, and drops the caller host and configured upstream headers from logged metadata.

BerriAI · 2026-08-23 · major

LiteLLM v1.98.0 — reserved capacity gets flat-cost billing, not per-token

LiteLLM v1.98.0 adds provisioned-throughput billing, so a deployment on reserved capacity carries a flat cost instead of a per-token charge. The release also ships shadow evals for the auto-router and a per-key prompt caching switch.

GitHub social preview card for the BerriAI/litellm repository

LiteLLM v1.98.0 teaches the open-source AI gateway to bill reserved capacity and to test routing changes before adopting them.

Quick facts

Version	v1.98.0 (stable)
Maker	BerriAI
Released	August 23, 2026
GitHub stars	57,066
Billing	Flat-cost PTU deployments, opt-in env var
Router	Shadow evals; groups callable as virtual models
Image	ghcr.io/berriai/litellm:v1.98.0, cosign-signed

What is it?

Provisioned-throughput (PTU) billing arrives in LiteLLM v1.98.0: you set a flat cost on a model deployment, and the gateway stops charging that deployment per token. Daily rollups record the flat cost by active hour, the model form and the Usage page surface it, and the whole feature sits behind an opt-in environment variable. LiteLLM is an open-source gateway that fronts more than 100 LLM APIs in the OpenAI format with cost tracking, guardrails, load balancing and logging.

How does it work?

The auto-router picks up a pre-adoption shadow eval: a blind pairwise judge samples live /v1/messages and /v1/responses traffic and scores a candidate routing change against the routing in use, including in the reverse direction, with results in a new shadow evals tab beside auto-router usage. Routing groups are now callable as virtual models and appear in /v1/models, so a client can ask for a group by name instead of a single deployment.

Why does it matter?

Platform teams that buy reserved capacity from a provider were mispricing it, because the gateway charged every call per token no matter how the capacity was bought. Flat-cost deployments fix that line in the bill. The rest of the release aims at long-running production traffic: a per-deployment keepalive_seconds heartbeat stops load balancers from cutting long streams, estimated output tokens become configurable per key, team and model for rate limiting, and prompt caching turns into a per-key switch.

Who is it for?

platform teams running a shared LLM gateway

Frequently asked questions

Is LiteLLM v1.98.0 a stable release or a preview?: LiteLLM v1.98.0 is published as a stable release on GitHub, not a pre-release, while the v1.99.0 line was still shipping release candidates and dev builds at the same time. The Docker image ghcr.io/berriai/litellm:v1.98.0 is signed with cosign, and the release notes give the verification command against a pinned commit hash.
How do I turn on provisioned-throughput billing in LiteLLM?: Flat-cost attribution in LiteLLM v1.98.0 is gated behind an opt-in environment variable, so it stays off until you switch it on. After that you enter the PTU inputs on the model deployment in the model form, and daily rollups write the per-model flat cost by active hour so the Usage page can show it.
What is a shadow eval in LiteLLM's auto-router?: A shadow eval lets a LiteLLM operator test a routing change before adopting it. The auto-router samples live /v1/messages and /v1/responses traffic and has a blind pairwise judge compare the candidate routing against the one in use, in either direction. A direction picker and reverse-mode display sit in the shadow evals tab.
Which new model pricing does LiteLLM v1.98.0 add?: LiteLLM v1.98.0 adds price entries for NVIDIA Nemotron 3.5 Lightning on both OpenRouter and DeepInfra. LiteLLM keeps a model-price map that drives its cost tracking, so a model needs an entry there before spend on it can be attributed.
Does LiteLLM v1.98.0 change anything for MCP servers?: Yes. LiteLLM v1.98.0 scopes gateway session bearer tokens to the RFC 8707 resource, so an MCP session token is tied to the resource it was issued for. The release also exposes client HTTP headers to logging callbacks and hooks, and drops the caller host and configured upstream headers from logged metadata.

Try it

docker pull ghcr.io/berriai/litellm:v1.98.0