Runta · 2026-11-17 · notable
FrontierHarness Eval v1.1 — which agent harness works best for each model
In short
FrontierHarness Eval v1.1 tests 22 coding-agent harnesses with 5 open models on 62 terminal tasks. The best harness changes a lot by model, and the model explains 92.8% of the cost differences.
Runta · 2026-11-17 · 4 sources checked

A 27,280-run leaderboard shows the same harness can rank first with one model and 16th with another.
What is it?
FrontierHarness Eval v1.1 is Runta's leaderboard for agent harnesses, the software that wraps a model and runs its tool calls. Version 1.1 covers 22 harness setups and 5 open models (GLM-5.3, GLM-5.3 Flash, DeepSeek V4.1 Flash, Muse and Kimi-K3) on 62 terminal tasks, with 4 attempts each. It is about 76 times larger than v1.0, which tested nine harnesses with Kimi K3 only.
How does it work?
Every harness ran through the Harbor framework with default settings at maximum reasoning effort, in a fresh Runta runtime per trial, with Baseten serving the open models. The team measured cost per task and cost per pass, flagged 216 reward-hacking trials as zero, and resampled the tasks 4,000 times to test how stable the rankings are.
Why does it matter?
The best harness depends on the model: Codex CLI is the cheapest per pass with GLM-5.3 ($0.803) but falls to 16th with Kimi-K3, and the median harness moves 11 of 22 places between models. Pi, Codex CLI and ZCode tied for the top pass rate at 61.9%. Runta notes that only open models were tested, so results for closed models may differ.
Who is it for?
teams choosing a coding-agent harness for open models
Try it
https://frontierharness.org/