# Coze Loop

> Prompt development, evaluation and tracing for AI agents in one platform

- Category: Evaluation & Production Ops / Observability & LLMOps
- Language: Go
- Licence: Apache-2.0
- Repository: https://github.com/coze-dev/coze-loop
- Website: https://www.coze.cn/loop
- Canonical page: https://ai-tldr.dev/tools/coze-loop/

## In short

Coze Loop is a developer-oriented platform for developing and operating AI agents, covering the lifecycle from prompt writing through debugging and evaluation to production monitoring. It comes from the Coze team and is the open-source edition of their commercial product, sharing the core framework so teams can self-host and extend it. It sits in the Observability & LLMOps category of the Evaluation & Production Ops tools and is open-source, written in Go and licensed Apache-2.0.

## Overview

Coze Loop is a developer-oriented platform for developing and operating AI agents, covering the lifecycle from prompt writing through debugging and evaluation to production monitoring. It comes from the Coze team and is the open-source edition of their commercial product, sharing the core framework so teams can self-host and extend it.

The platform is organised around three modules that usually live in three separate tools. Prompt development gives you a visual playground for interactive testing, side-by-side comparison of different LLMs' output, and version management of the prompts themselves. Evaluation adds managed evaluation sets, configurable evaluators and experiments, so prompt and agent output can be scored automatically on dimensions like accuracy, conciseness and compliance instead of eyeballed.

Observability closes the loop. An SDK reports traces covering the whole execution path — prompt parsing, model invocation, tool execution — capturing intermediate results and exceptions along the way, so a regression seen in production can be traced back to the prompt version that caused it. Coze Loop is written in Go, ships Docker Compose and Helm deployments, and integrates with OpenAI, Volcengine Ark and other model providers.

## What it does

- Prompt playground for real-time interactive testing and side-by-side comparison of output across different LLMs
- Prompt version management, so a production regression traces back to a specific revision
- Evaluation module with managed evaluation sets, evaluators and experiments for automated multi-dimensional scoring
- Trace observability across the full execution path — prompt parsing, model calls and tool execution — with intermediate results and exceptions captured
- SDK-based trace reporting, plus integration with OpenAI, Volcengine Ark and other model providers
- Self-hosted via Docker Compose or a Kubernetes Helm chart

## Getting started

Coze Loop is deployed rather than installed as a library. Docker Compose is the quickest route; a Helm chart is provided for Kubernetes. You configure at least one model before starting the service.

1. Clone the repository

   Deployment configuration lives in the repo, so start by cloning it.

```bash
git clone https://github.com/coze-dev/coze-loop.git
cd coze-loop
```

2. Configure a model

   Edit release/deployment/docker-compose/conf/model_config.yaml and set the api_key and model fields for your provider — for Volcengine Ark, model is the endpoint ID of the model access point.

```yaml
# release/deployment/docker-compose/conf/model_config.yaml
api_key: <your provider api key>
model: <model or endpoint id>
```

3. Start the service

   One make target brings up the Docker Compose stack in development mode.

```bash
make compose-up
```

4. Open the console

   Browse to the local instance and start creating prompts, evaluation sets and experiments. For Kubernetes, deploy the Helm chart instead — the quickstart wiki covers a Minikube walkthrough.

```text
http://localhost:8082
```

## When to use it

- Compare how several models answer the same prompt before committing one to production, with the comparison recorded rather than ad hoc
- Score an agent's output automatically on accuracy, conciseness or compliance whenever a prompt changes
- Trace a bad production answer back through tool calls and model invocations to the step that went wrong
- Self-host the whole prompt-eval-observability loop when traces and prompts cannot be sent to a hosted LLMOps vendor