Kong Gateway

// Overview

Kong Gateway is a cloud-native, platform-agnostic API gateway built on NGINX and extended through Lua plugins. Its core job is the classic one — proxying, routing, load balancing, health checking, authentication with JWT, basic auth, OAuth and ACLs, rate limiting, request and response transformation, TLS termination — for L4 and L7 traffic. It runs natively on Kubernetes through an official ingress controller, and supports declarative database-less deployment as well as hybrid control-plane/data-plane splits.

What puts it in an AI tooling directory is the AI Gateway layer built on the same plugin machinery. Kong exposes a universal LLM API that routes across providers including OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure AI, Databricks, Mistral and Hugging Face, so an application talks to one endpoint instead of a dozen SDKs. On top of that sit AI-specific plugins the project counts at 60-plus: AI observability, semantic security, semantic caching and semantic routing.

Kong also treats MCP as first-class traffic: it can govern, secure and observe MCP calls, and autogenerate an MCP server from any RESTful API. The gateway is Apache-2.0 licensed and self-hostable; Kong Inc. sells commercial subscriptions around it, notably the Konnect managed control plane with analytics, service catalog and developer portals. Plugins can be written in Lua, Go or JavaScript through the Plugin Development Kit.

// What it does

• Universal LLM API that routes one request format across OpenAI, Anthropic, Gemini, Bedrock, Azure AI, Databricks, Mistral, Hugging Face and more
• AI plugins for observability, semantic security, semantic caching and semantic routing
• MCP traffic governance, security and observability, plus MCP autogeneration from a REST API
• Full conventional gateway feature set: routing, load balancing, health checks, auth, rate limiting, transformations
• Declarative DB-less and hybrid control-plane/data-plane deployments; native Kubernetes ingress controller
• Plugin Hub plus a PDK for writing your own plugins in Lua, Go or JavaScript

// Getting started

The README's suggested path is the docker-compose distribution; a DB-less Docker install is also documented. Once the gateway is up, the AI Gateway capabilities are configured as plugins.

git clone https://github.com/Kong/docker-kong
cd docker-kong/compose/

KONG_DATABASE=postgres docker-compose --profile database up

:8000  proxy
:8001  Admin API
:8002  Kong Manager UI

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Put one authenticated, rate-limited endpoint in front of several LLM providers so applications stop embedding provider SDKs
• Apply the same governance to AI traffic as to the rest of your API estate — auth, quotas, logging, tracing
• Cache and route requests semantically to cut spend on repeated or low-difficulty prompts
• Expose and police MCP servers, or generate one from an existing REST API

// How Kong Gateway compares

Kong Gateway alongside other open-source gateways & routing tools AI/TLDR tracks, ranked by GitHub stars.