# Apache APISIX

> Cloud-native API gateway with AI plugins for proxying and governing LLM traffic

- Category: Evaluation & Production Ops / Gateways & Routing
- Language: Lua
- Licence: Apache-2.0
- Repository: https://github.com/apache/apisix
- Website: https://apisix.apache.org/ai-gateway/
- Canonical page: https://ai-tldr.dev/tools/apache-apisix/

## In short

Apache APISIX is a dynamic, real-time API gateway that handles traffic management tasks such as load balancing, dynamic upstreams, canary releases, circuit breaking, authentication, and observability. It runs anywhere from bare-metal to Kubernetes and can also act as a Kubernetes ingress controller. It sits in the Gateways & Routing category of the Evaluation & Production Ops tools and is open-source, written in Lua and licensed Apache-2.0.

## Overview

Apache APISIX is a dynamic, real-time API gateway that handles traffic management tasks such as load balancing, dynamic upstreams, canary releases, circuit breaking, authentication, and observability. It runs anywhere from bare-metal to Kubernetes and can also act as a Kubernetes ingress controller.

Through its plugin system, APISIX works as an AI gateway: it proxies requests to LLM providers, load balances across them, applies retries and fallbacks, and enforces token-based rate limiting and security controls. An mcp-bridge plugin can also expose stdio-based MCP servers as HTTP SSE services.

It fits the LLM gateway / proxy category for teams that already need a general-purpose gateway and want one place to route both traditional API traffic and AI model traffic, without locking into a single vendor.

## What it does

- AI proxying and load balancing across LLM upstreams, with retries and fallbacks
- Token-based rate limiting plus authentication and security plugins for AI traffic
- `mcp-bridge` plugin converts stdio-based MCP servers into scalable HTTP SSE services
- Hot updates of configuration and plugins without restarts
- Multi-protocol support: HTTP(S), gRPC, TCP/UDP, MQTT, Dubbo, WebSocket, and HTTP/3 with QUIC
- Fine-grained routing with full-path and prefix matching, plus health checks and circuit breaking

## Getting started

Run APISIX locally with the official quickstart script, then create and test a route through the Admin API.

1. Start APISIX

   The quickstart script starts two Docker containers, apisix-quickstart and etcd. Docker is required.

```bash
curl -sL https://run.api7.ai/apisix/quickstart | sh
```

2. Verify it is running

   Check that APISIX answers on its proxy port 9080.

```bash
curl "http://127.0.0.1:9080" --head | grep Server
```

3. Create a route

   Use the Admin API on port 9180 to forward /ip to an upstream.

```bash
curl -i "http://127.0.0.1:9180/apisix/admin/routes" -X PUT -d '{
  "id": "getting-started-ip",
  "uri": "/ip",
  "upstream": {
    "type": "roundrobin",
    "nodes": { "httpbin.org:80": 1 }
  }
}'
```

4. Test the route

   Send a request through the gateway on port 9080 to confirm the route works.

```bash
curl "http://127.0.0.1:9080/ip"
```

## When to use it

- Put a single gateway in front of multiple LLM providers, with load balancing, retries, and fallbacks between them
- Enforce token-based rate limits, authentication, and access control on AI model traffic
- Expose stdio-based MCP servers as HTTP SSE services using the mcp-bridge plugin
- Run one gateway for both traditional north-south API traffic and AI traffic, including as a Kubernetes ingress controller