# Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents

> Grading AI agents on what they did, not just what they said — and discovering final-output grading misses 44 percent of safety violations.

- Org: Peking University, HKU
- Date: 2026-04-07
- Category: benchmark, paper, repo
- Source: https://arxiv.org/abs/2604.06132
- Canonical page: https://ai-tldr.dev/releases/claw-eval-trustworthy-agent-eval/

## In short

A 300-task benchmark that grades agents on the full trajectory — execution traces, audit logs and environment snapshots — not just final outputs. Catches 44% of safety violations final-output grading misses.

## What it is

Claw-Eval is a new evaluation framework for autonomous agents from researchers at Peking University and the University of Hong Kong. It argues the standard way of grading agents — checking only the final output — is 'trajectory-opaque' and systematically unreliable. Instead Claw-Eval records the whole trajectory through three independent channels: execution traces, audit logs and environment snapshots.

## How it works

The suite contains 300 human-verified tasks in nine categories across three groups — general service orchestration, multimodal perception and generation, and multi-turn professional dialogue — with 2,159 fine-grained rubric items. Each task is graded on completion, safety and robustness across three trials, with the three evidence channels cross-checked against each other. The authors evaluated 14 frontier models and found trajectory-opaque evaluation misses 44% of safety violations and 13% of robustness failures that Claw-Eval's full-trajectory grading catches.

## Why it matters

Every agent benchmark that only checks the final answer is under-reporting risk. Claw-Eval gives teams a concrete, reproducible way to catch agents that 'get the right answer the wrong way' — especially important as autonomous agents start touching real systems.

## Who it's for

Agent framework authors, red-teamers, anyone shipping agents to production.

## Links

- [Paper](https://arxiv.org/abs/2604.06132)
- [Repo](https://github.com/claw-eval/claw-eval)
- [Leaderboard](https://claw-eval.github.io/)