OmniRoute

// Overview

OmniRoute is an MIT-licensed AI gateway that you install and run on your own machine. It aggregates hundreds of model providers — the major labs (OpenAI, Anthropic, Google Gemini, xAI, DeepSeek, Mistral, Qwen, Meta Llama, Groq, NVIDIA) alongside many smaller ones, a large share of which have free tiers — and exposes all of them through a single OpenAI-compatible `/v1` endpoint on `localhost:20128`, with a web dashboard for managing credentials and routing.

The point of the aggregation is resilience and cost. Requests can be sent to the special `auto` model and OmniRoute picks a provider for you; when one fails or hits a quota, quota-aware fallback moves the request on, backed by circuit breakers, exponential backoff and per-connection cooldowns. A multi-engine compression pipeline (which bundles techniques such as RTK, Caveman and LLMLingua-2) can shrink prompts before they go out, and a set of routing strategies covers priority, round-robin, cost-optimised, auto-scoring, fusion and pipeline modes.

It is aimed squarely at coding agents: the project lists integrations with Claude Code, Cursor, Cline, Codex, GitHub Copilot, Continue, Aider, OpenCode and other CLI tools, all of which just need their base URL pointed at the local gateway. Credentials are stored locally with AES-256-GCM encryption and telemetry is off by default; MCP and A2A support lets an agent drive the gateway itself. It is written in TypeScript and ships via npm, Docker, pnpm, AUR and Nix.

// What it does

• One OpenAI-compatible endpoint in front of hundreds of providers and models, including many free tiers
• Quota-aware automatic fallback with circuit breakers, exponential backoff and per-connection cooldowns
• A range of routing strategies — priority, round-robin, cost-optimised, auto-scoring, fusion and pipeline
• A multi-engine prompt-compression pipeline (RTK, Caveman, LLMLingua-2 and others) to cut outbound tokens
• Zero-config `auto` model selection, so a client can work before any provider key is added
• MCP and A2A support so agents can control the gateway as a tool
• Local-first by design: AES-256-GCM encrypted credentials and no telemetry by default

// Getting started

OmniRoute installs as a global npm package and runs as a local server with a dashboard. Docker, pnpm, AUR and Nix packages are also available.

npm install -g omniroute
omniroute

curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'

http://localhost:20128/v1

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Give a coding agent one stable endpoint and let the gateway handle provider outages and quota limits
• Cut spend by routing cheap requests to free or low-cost providers and compressing prompts before they leave
• Try many models from many providers without wiring up a separate SDK and key for each one
• Keep keys and traffic on your own machine when a hosted gateway is not acceptable

// How OmniRoute compares

OmniRoute alongside other open-source gateways & routing tools AI/TLDR tracks, ranked by GitHub stars.