Weco

// Overview

Weco is a command-line tool that systematically optimizes code against an evaluation metric you define. You give it a file to change, a script that scores the result and a goal; it runs an LLM-guided tree search that proposes edits, applies them, runs your evaluation, parses the score and uses that to decide where to search next. The loop is the AIDE method from Weco AI's paper, packaged as a tool you point at your own repository.

What it optimizes is deliberately broad, because the only requirement is a measurable objective. The project's own examples cover agent and harness engineering (prompts, tools and agentic scaffolding scored on accuracy, win rate or cost), prompt engineering and prompt compression, model development from feature transformations up to a whole training pipeline scored on validation accuracy, AUC or Sharpe ratio, and raw performance work on CUDA and Triton GPU kernels scored on latency, throughput or memory bandwidth.

Two things make it usable inside an existing workflow. Weco ships as a skill for AI coding assistants — `weco setup claude-code` or `weco setup cursor` installs instructions that teach the assistant to write the evaluation, configure the run and interpret the results from a plain-language request. And `weco observe` decouples tracking from optimizing: if you are running your own loop with an agent or a script, you can log steps and metrics to the Weco dashboard for tree visualization and code diffs without handing over control. The observe commands are built to sit inside agent loops — Weco-side blips are retried and reported rather than fatal, unless you pass `--strict`. The CLI is Apache-2.0 on PyPI; the hosted dashboard is the commercial side.

// What it does

• LLM-guided tree search that edits code, runs your evaluation script, reads the score and iterates
• Optimizes anything with a measurable objective — prompts, agent scaffolds, training pipelines, CUDA and Triton kernels
• Ships as a skill for Claude Code and Cursor, so an assistant can drive the whole optimization from a plain-language request
• `weco observe` tracks external experiments — your own loop, logged to the dashboard with tree visualization, code diffs and metric tracking
• Agent-safe exit codes: observe logging fails loudly only on errors you must fix, and retries transient ones unless `--strict` is set
• Python SDK alongside the CLI; installs with `pip install weco`

// Getting started

Weco needs Python 3.8 or newer. The examples directory in the repository carries runnable end-to-end walkthroughs, including a Colab notebook.

pip install weco

weco setup claude-code   # installs the skill into Claude Code
weco setup cursor        # installs the skill into Cursor

WECO_RUN_ID=$(weco observe init --name "my-experiment" --metric val_bpb --goal min --source train.py)

weco observe log --run-id "$WECO_RUN_ID" --step 0 --description "baseline" \
  --metrics '{"val_bpb": 2.36}' --source train.py
weco observe log --run-id "$WECO_RUN_ID" --step 1 --description "increase batch size" \
  --metrics '{"val_bpb": 2.26}' --source train.py

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Tuning an agent harness — prompts, tools, scaffolding — against a score instead of by intuition
• Squeezing latency or throughput out of a CUDA or Triton kernel with a benchmark as the objective
• Sweeping a training pipeline's transformations and architecture against validation accuracy or AUC
• Keeping a visual record of an optimization loop you run yourself, via `weco observe`

// How Weco compares

Weco alongside other open-source prompt programming tools AI/TLDR tracks, ranked by GitHub stars.