# FrontierHarness Eval v1.1 — which agent harness works best for each model

> A 27,280-run leaderboard shows the same harness can rank first with one model and 16th with another.

- Org: Runta
- Date: 2026-11-17
- Category: benchmark
- Source: https://runta.com/blog/introducing-frontierharness-eval-v1-1/
- Canonical page: https://ai-tldr.dev/releases/runta-frontierharness-eval-v1-1/

## In short

FrontierHarness Eval v1.1 tests 22 coding-agent harnesses with 5 open models on 62 terminal tasks. The best harness changes a lot by model, and the model explains 92.8% of the cost differences.

## What it is

FrontierHarness Eval v1.1 is Runta's leaderboard for agent harnesses, the software that wraps a model and runs its tool calls. Version 1.1 covers 22 harness setups and 5 open models (GLM-5.3, GLM-5.3 Flash, DeepSeek V4.1 Flash, Muse and Kimi-K3) on 62 terminal tasks, with 4 attempts each. It is about 76 times larger than v1.0, which tested nine harnesses with Kimi K3 only.

## How it works

Every harness ran through the Harbor framework with default settings at maximum reasoning effort, in a fresh Runta runtime per trial, with Baseten serving the open models. The team measured cost per task and cost per pass, flagged 216 reward-hacking trials as zero, and resampled the tasks 4,000 times to test how stable the rankings are.

## Why it matters

The best harness depends on the model: Codex CLI is the cheapest per pass with GLM-5.3 ($0.803) but falls to 16th with Kimi-K3, and the median harness moves 11 of 22 places between models. Pi, Codex CLI and ZCode tied for the top pass rate at 61.9%. Runta notes that only open models were tested, so results for closed models may differ.

## Who it's for

teams choosing a coding-agent harness for open models

## Try it

https://frontierharness.org/

## Links

- [Announcement](https://runta.com/blog/introducing-frontierharness-eval-v1-1/)
- [Leaderboard](https://frontierharness.org/)
- [Harbor Framework](https://github.com/harbor-framework/harbor)
- [HN Discussion](https://news.ycombinator.com/item?id=50532191)
