Coze Loop

// Overview

Coze Loop is a developer-oriented platform for developing and operating AI agents, covering the lifecycle from prompt writing through debugging and evaluation to production monitoring. It comes from the Coze team and is the open-source edition of their commercial product, sharing the core framework so teams can self-host and extend it.

The platform is organised around three modules that usually live in three separate tools. Prompt development gives you a visual playground for interactive testing, side-by-side comparison of different LLMs' output, and version management of the prompts themselves. Evaluation adds managed evaluation sets, configurable evaluators and experiments, so prompt and agent output can be scored automatically on dimensions like accuracy, conciseness and compliance instead of eyeballed.

Observability closes the loop. An SDK reports traces covering the whole execution path — prompt parsing, model invocation, tool execution — capturing intermediate results and exceptions along the way, so a regression seen in production can be traced back to the prompt version that caused it. Coze Loop is written in Go, ships Docker Compose and Helm deployments, and integrates with OpenAI, Volcengine Ark and other model providers.

// What it does

• Prompt playground for real-time interactive testing and side-by-side comparison of output across different LLMs
• Prompt version management, so a production regression traces back to a specific revision
• Evaluation module with managed evaluation sets, evaluators and experiments for automated multi-dimensional scoring
• Trace observability across the full execution path — prompt parsing, model calls and tool execution — with intermediate results and exceptions captured
• SDK-based trace reporting, plus integration with OpenAI, Volcengine Ark and other model providers
• Self-hosted via Docker Compose or a Kubernetes Helm chart

// Getting started

Coze Loop is deployed rather than installed as a library. Docker Compose is the quickest route; a Helm chart is provided for Kubernetes. You configure at least one model before starting the service.

git clone https://github.com/coze-dev/coze-loop.git
cd coze-loop

# release/deployment/docker-compose/conf/model_config.yaml
api_key: <your provider api key>
model: <model or endpoint id>

make compose-up

http://localhost:8082

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Compare how several models answer the same prompt before committing one to production, with the comparison recorded rather than ad hoc
• Score an agent's output automatically on accuracy, conciseness or compliance whenever a prompt changes
• Trace a bad production answer back through tool calls and model invocations to the step that went wrong
• Self-host the whole prompt-eval-observability loop when traces and prompts cannot be sent to a hosted LLMOps vendor

// How Coze Loop compares

Coze Loop alongside other open-source observability & llmops tools AI/TLDR tracks, ranked by GitHub stars.