# Opik

> Open-source tracing, evaluation, and monitoring for LLM and agent apps

- Category: Evaluation & Production Ops / Observability & LLMOps
- Language: Python
- Licence: MIT
- Repository: https://github.com/comet-ml/opik
- Website: https://www.comet.com/site/products/opik
- Canonical page: https://ai-tldr.dev/tools/opik/

## In short

Opik is an open-source platform from Comet for building, testing, and monitoring applications built on large language models. It records the calls your app makes to an LLM, stores them as traces, and lets you inspect what went in and what came back, from early prototyping through production. It sits in the Observability & LLMOps category of the Evaluation & Production Ops tools and is open-source, written in Python and licensed MIT.

## Overview

Opik is an open-source platform from Comet for building, testing, and monitoring applications built on large language models. It records the calls your app makes to an LLM, stores them as traces, and lets you inspect what went in and what came back, from early prototyping through production.

It is aimed at developers and teams working on RAG chatbots, code assistants, and agent systems who need to see why an app behaves the way it does. Alongside tracing, it offers evaluation tools, including LLM-as-a-judge metrics for things like hallucination detection, moderation, and RAG answer relevance.

As an LLM observability tool, Opik pairs a self-hostable server (run locally with Docker Compose) with client SDKs and many framework integrations, so you can collect traces, score them, and watch feedback scores, trace counts, and token usage over time in a dashboard.

## What it does

- Tracing of LLM calls, conversations, and agent activity, viewable in a UI
- LLM-as-a-judge metrics for hallucination detection, moderation, and RAG assessment (answer relevance, context precision)
- Datasets and experiments to automate evaluation, plus a PyTest integration for CI/CD
- Production monitoring dashboards with online evaluation rules; built to handle high trace volumes
- Many third-party framework integrations, including Google ADK, Autogen, and Flowise AI
- Self-hostable server via Docker Compose, plus the Opik Agent Optimizer and Guardrails

## Getting started

Install the Python SDK, optionally run the server locally, then wrap your LLM function with a decorator to start logging traces.

1. Install the SDK

   Install the Opik client with pip (or uv).

```bash
pip install opik
```

2. Run the server locally (optional)

   Clone the repo and start the self-hosted server with the install script; the UI runs at http://localhost:5173.

```bash
git clone https://github.com/comet-ml/opik.git
cd opik
./opik.sh
```

3. Configure

   Point the SDK at your local server (or Comet cloud).

```bash
opik configure
```

4. Trace a function

   Add the @opik.track decorator to log calls as traces.

```python
import opik

opik.configure(use_local=True)

@opik.track
def my_llm_function(user_question: str) -> str:
    # Your LLM code here
    return "Hello"
```

## When to use it

- Debugging a RAG chatbot by inspecting each LLM call and its retrieved context in traces
- Scoring outputs for hallucinations or moderation issues using LLM-as-a-judge metrics
- Adding evaluation checks to a CI/CD pipeline with the PyTest integration
- Monitoring trace counts, token usage, and feedback scores for an agent app in production