What does the classifier circuit breaker in LiteLLM v1.101.0 protect against?

LiteLLM's complexity router runs a classifier to judge how hard a request is. The new circuit breaker handles the case where that classifier fails or degrades, so routing falls back gracefully instead of stalling. A companion change auto-escalates complexity-router tasks that have already stalled.

Does LiteLLM v1.101.0 change how spending is tracked?

Yes. LiteLLM v1.101.0 adds time-based off-peak pricing support across multiple providers, so a request billed at a cheaper hour is costed at that rate. Cost tracking is also extended to guardrails and specialized services, and DataDog LLM observability gains cost dimensions.

What security controls are new in LiteLLM v1.101.0?

This release adds configurable password policies and an SSO-only login option for the proxy, restrictions on file uploads at /v1/files and /upload/logo, and workload identity federation through OIDC token exchange for OpenAI. Per-worker admission control also rejects excess requests with a 503 instead of queueing them.

Why move LiteLLM's Prometheus metrics to a separate port?

LiteLLM v1.101.0 serves Prometheus metrics from a separate process via --prometheus_metrics_port. Splitting the scrape endpoint off the main proxy process keeps metric collection from competing with request serving, and the release exposes new per-key and per-team rate-limit gauges on it.

BerriAI · 2026-09-15 · major

LiteLLM v1.101.0 — smarter complexity routing and a semantic MCP search

LiteLLM v1.101.0 lands 1,902 commits on the AI gateway: a second-generation heuristic complexity router, a classifier circuit breaker, semantic tool search for the native MCP Gateway, and off-peak pricing in cost tracking.

The gateway gets a second-generation complexity router, a circuit breaker for its classifier, and semantic search over MCP tools.

Quick facts

Project	LiteLLM, by BerriAI
Version	v1.101.0
Released	15 September 2026
Commits since v1.100.0	1,902
Routing	Heuristic v2 complexity routing
MCP	Semantic tool search in the native MCP Gateway
Container base	glibc 2.44, Python 3.13

What is it?

Heuristic v2 complexity routing headlines LiteLLM v1.101.0. The router decides which model a request should go to based on how hard the request looks, and this version also adds a classifier circuit breaker for when that decision path fails, auto-escalation for stalled complexity-router tasks, and opt-in modality-based capability routing so image requests reach a model that can actually see.

How does it work?

LiteLLM sits in front of 100+ provider APIs and speaks OpenAI format to your code. The native MCP Gateway gained semantic tool search, which matters once a gateway fronts more tools than fit in one prompt. On the operations side, Prometheus metrics move to a separate process behind --prometheus_metrics_port, per-key and per-team rate-limit gauges are exposed, OpenTelemetry spans now carry the routing decision, and a new /v1/responses/input_tokens endpoint counts tokens before you spend them.

Why does it matter?

This is the release where the gateway's admin surface catches up with the routing surface. Cost tracking learns time-based off-peak pricing across several providers. Access control learns configurable password policies and SSO-only login, and file uploads are restricted at /v1/files and /upload/logo. OpenAI calls can now authenticate through workload identity federation using OIDC token exchange, so a deployment can drop a long-lived API key.

Who is it for?

platform teams running a shared LLM gateway

Frequently asked questions

What does the classifier circuit breaker in LiteLLM v1.101.0 protect against?: LiteLLM's complexity router runs a classifier to judge how hard a request is. The new circuit breaker handles the case where that classifier fails or degrades, so routing falls back gracefully instead of stalling. A companion change auto-escalates complexity-router tasks that have already stalled.
Does LiteLLM v1.101.0 change how spending is tracked?: Yes. LiteLLM v1.101.0 adds time-based off-peak pricing support across multiple providers, so a request billed at a cheaper hour is costed at that rate. Cost tracking is also extended to guardrails and specialized services, and DataDog LLM observability gains cost dimensions.
What security controls are new in LiteLLM v1.101.0?: This release adds configurable password policies and an SSO-only login option for the proxy, restrictions on file uploads at /v1/files and /upload/logo, and workload identity federation through OIDC token exchange for OpenAI. Per-worker admission control also rejects excess requests with a 503 instead of queueing them.
Why move LiteLLM's Prometheus metrics to a separate port?: LiteLLM v1.101.0 serves Prometheus metrics from a separate process via --prometheus_metrics_port. Splitting the scrape endpoint off the main proxy process keeps metric collection from competing with request serving, and the release exposes new per-key and per-team rate-limit gauges on it.

Try it

pip install 'litellm[proxy]'