# HAL — Princeton's Holistic Agent Leaderboard Accepted at ICLR 2026, Now Tracks 26K+ Rollouts

> HAL standardizes agent evaluation across 9 benchmarks with cost-tracking — revealing 100x cost differentials for 1% accuracy gains.

- Org: Princeton University
- Date: 2026-04-23
- Category: benchmark
- Source: https://hal.cs.princeton.edu/
- Canonical page: https://ai-tldr.dev/releases/hal-holistic-agent-leaderboard-iclr-2026/

## In short

The Holistic Agent Leaderboard, accepted at ICLR 2026, provides standardized cost-aware agent evaluation across 9 benchmarks covering coding, web navigation, science tasks, and customer service. It now tracks 26,597 rollouts and has shifted focus to multi-dimensional reliability measurement.

## What it is

The Holistic Agent Leaderboard (HAL) from Princeton's SAgE team was accepted as a conference paper at ICLR 2026 and presented at the Rio de Janeiro conference in late April. It provides a unified, open-source harness for reproducible, cost-controlled agent benchmarking across multiple real-world task domains.

## How it works

HAL orchestrates parallel evaluations across hundreds of VMs, reducing evaluation time from weeks to hours. It covers 9 distinct benchmarks spanning web assistance (AssistantBench, GAIA, Online Mind2Web), scientific programming (CORE-Bench, Scicode, ScienceAgentBench), software engineering (SWE-bench Verified Mini), customer service (TAU-bench Airline), and programming competitions (USACO). Pareto frontier visualizations reveal cost-performance tradeoffs across agents.

## Why it matters

HAL's key finding is that agents can be 100x more expensive while only 1% better on accuracy, which leaderboards without cost-reporting completely hide. The platform has also shifted focus to reliability measurement — showing that agent performance can drop from 60% to 25% under multi-run consistency checks. With 26,597 rollouts logged, it is becoming the reference infrastructure for third-party agent evaluations.

## Links

- [HAL Website](https://hal.cs.princeton.edu/)
- [arXiv Paper](https://arxiv.org/abs/2510.11977)
- [ICLR 2026](https://openreview.net/forum?id=vUaY1t64ZZ)