# Arize Phoenix

> Open-source observability and evaluation for LLM and agent apps

- Category: Evaluation & Production Ops / Evaluation & Red-Teaming
- Language: Python
- Licence: Elastic-2.0
- Repository: https://github.com/Arize-ai/phoenix
- Website: https://phoenix.arize.com
- Canonical page: https://ai-tldr.dev/tools/arize-phoenix/

## In short

Arize Phoenix is an open-source AI observability platform for experimenting with, evaluating, and troubleshooting LLM and agent applications. It traces your app's runtime with OpenTelemetry-based instrumentation, then lets you run evaluations on those traces to find where things go wrong. It sits in the Evaluation & Red-Teaming category of the Evaluation & Production Ops tools and is open-source, written in Python and licensed Elastic-2.0.

## Overview

Arize Phoenix is an open-source AI observability platform for experimenting with, evaluating, and troubleshooting LLM and agent applications. It traces your app's runtime with OpenTelemetry-based instrumentation, then lets you run evaluations on those traces to find where things go wrong.

It is built for developers and ML teams who need to see what their LLM application is actually doing in development and production. Phoenix is vendor and language agnostic, with out-of-the-box support for frameworks like LangGraph, LlamaIndex, DSPy, CrewAI, the OpenAI Agents SDK, and the Claude Agent SDK, plus providers such as OpenAI, Anthropic, Google GenAI, and AWS Bedrock.

As an eval framework, it pairs tracing with response and retrieval evals, versioned datasets, and experiments so you can measure changes to prompts, models, and retrieval. It runs on your local machine, in a Jupyter notebook, in a container, or in the cloud.

## What it does

- Tracing of LLM and agent runtime using OpenTelemetry-based instrumentation
- LLM-based evaluation with response and retrieval evals to benchmark performance
- Versioned datasets of examples for experimentation, evaluation, and fine-tuning
- Experiments to track and compare changes to prompts, LLMs, and retrieval
- Prompt playground and prompt management with version control, tagging, and replay
- Auto-instrumentation for popular frameworks and LLM providers via OpenInference

## Getting started

Install the Phoenix package, start the local server, then point your application's instrumentation at it.

1. Install Phoenix

   Install the full Phoenix platform from PyPI. A conda-forge build is also available.

```bash
pip install arize-phoenix
```

2. Start the Phoenix server

   Launch the server from your terminal. The Phoenix UI opens at http://localhost:6006, where you can view traces and run evals.

```bash
phoenix serve
```

3. Send traces from your app

   Instrument your LLM or agent code with OpenInference auto-instrumentation so spans flow into Phoenix. See the docs for the integration matching your framework or provider (OpenAI, Anthropic, LangGraph, LlamaIndex, and others).

## When to use it

- Debugging an agent that calls tools by tracing each step and inspecting where a run fails
- Running response and retrieval evals on a RAG pipeline to measure answer and retrieval quality
- Comparing prompt or model changes across versioned datasets using experiments before shipping
- Monitoring LLM application behavior in production to catch regressions and troubleshoot issues