Petri

// Overview

Petri is an alignment-auditing agent for language models. You give it seed instructions describing a behaviour you are worried about — sycophancy, reward hacking, deception — and it generates realistic audit scenarios, runs multi-turn conversations against the model under test, simulates tools and rollbacks so the model can actually act, and then scores the resulting transcripts against a consistent rubric.

Three model roles do the work, and they are configured separately: the **auditor** designs and drives the audit, the **target** is the model being evaluated, and the **judge** scores the transcript across behavioural dimensions. Splitting the auditor from the target means the target can be swapped — or wrapped in its own agent scaffold — without rewriting the audit logic.

Petri began as work by the Alignment team at Anthropic and is now developed by the nonprofit Meridian Labs, in collaboration with the UK AISI Red Team. It is built on Inspect AI and follows standard Inspect conventions, so `inspect eval` and `inspect view` work exactly as they do for any other Inspect task, and it sits alongside Inspect Scout and Inspect Flow in Meridian's stack.

// What it does

• Generates realistic multi-turn audit scenarios from short seed instructions, including tagged sets such as `tags:sycophancy`
• Separate auditor, target and judge model roles, each pointed at a different provider or model
• Simulated tools and rollbacks, so the audit tests what a model does and not only what it says
• Judge scoring against a consistent rubric across behavioural dimensions
• Built on Inspect AI — the same `inspect eval` / `inspect view` commands, flags and transcript viewer
• Bloom uses Petri as its execution backbone, pairing Petri's broad exploration with Bloom's graded single-behaviour assessments

// Getting started

Petri installs as an Inspect AI extension and is driven with the standard `inspect eval` command. You need API keys for whichever providers back the auditor, target and judge roles.

pip install inspect-petri

# or from the repository
pip install git+https://github.com/meridianlabs-ai/inspect_petri

export ANTHROPIC_API_KEY=...
export OPENAI_API_KEY=...

inspect eval inspect_petri/audit \
   -T seed_instructions=tags:sycophancy \
   --model-role auditor=anthropic/claude-sonnet-4-6 \
   --model-role target=openai/gpt-5-mini \
   --model-role judge=anthropic/claude-opus-4-6

inspect view

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Probe a model you are about to deploy for sycophancy, deception or reward hacking before it reaches users
• Re-run the same audit across model versions to see whether a behaviour got better or worse
• Audit a model inside its real agent scaffold, with its actual system prompt and tool definitions
• Pair broad exploratory audits with Bloom's deeper graded assessment of a single behaviour

// How Petri compares

Petri alongside other open-source evaluation & red-teaming tools AI/TLDR tracks, ranked by GitHub stars.