Can I run the full DAYJOB: Tax benchmark myself?

DAYJOB: Tax is mostly held out. Surge AI published only 2 of the 50 tasks as public samples on Hugging Face under the MIT license, so outside teams can see the format. Access to the full benchmark has to be requested from Surge AI by email at benchmarks@surgehq.ai, which keeps the remaining 48 tasks out of training data.

How do GPT-6 Astra and Gemini 4 Argon compare with Claude on DAYJOB: Tax?

On the DAYJOB: Tax leaderboard, Claude Sonnet 5.5, Claude Fable 5.1 and Claude Opus 5.5 take the top three spots at 10.8%, 9.6% and 9.2%. Grok 4.7 scores 5%, GPT-6 Astra 4.8%, GPT-6.1 Sol 4% and Gemini 4 Argon 3.2%. DeepSeek V4.1 Flash scores 1.2%, and Kimi K3, Mistral Large 4 and Qwen 3.8 Max pass no attempts at all.

What kinds of mistakes do AI agents make on DAYJOB: Tax?

Surge AI's DAYJOB: Tax write-up describes judgment errors rather than bad math. In a condo-sale case, an agent compared two closing dates correctly but missed a deadline that could end the seller's exclusion; Claude Opus caught it in all five attempts and GPT-6 Astra never did. In a separating-couple case, Opus kept assuming the couple was unmarried, while Astra checked marital status in four of five attempts.

How big are the DAYJOB: Tax assignments?

Each DAYJOB: Tax assignment comes with a realistic client workspace. The 50 tasks hold 1,789 files in total, mostly PDFs plus Excel workbooks and Word documents, with a median of about 30 files per task and up to 123. The public dataset card adds that each task has a median of 30.5 rubric criteria the agent's work is graded against.

Surge AI · 2026-10-27 · major

DAYJOB: Tax — best AI agent passes only 10.8% of real tax assignments

DAYJOB: Tax is Surge AI's new benchmark of 50 professional tax assignments with 1,789 client files. Claude Sonnet 5.5 leads at 10.8%, GPT-6 Astra scores 4.8% and Gemini 4 Argon 3.2%.

Cover image for Surge AI's DAYJOB: Tax benchmark of AI agents on professional tax work

Expert-built tax jobs, from a client's request to the finished work: no frontier agent passes more than about one in ten.

Key specs

Top pass rate	10.8%
Assignments	50

Quick facts

Made by	Surge AI
Tasks	50 tax assignments in 8 practice areas
Workspace files	1,789 (median 30 per task, up to 123)
Pass rule	Meet at least 95% of the task's rubric
Leader	Claude Sonnet 5.5 — 10.8%
Public data	2 sample tasks on Hugging Face (MIT)

Benchmarks

DAYJOB: Tax pass rate (95% rubric threshold)

Claude Sonnet 5.5		10.8%
Claude Fable 5.1		9.6%
Claude Opus 5.5		9.2%
Muse Wonder 1.0		6%
Grok 4.7		5%
GPT-6 Astra		4.8%
GPT-6.1 Sol		4%
Gemini 4 Argon		3.2%
Kimi K3		0%

source ↗

What is it?

DAYJOB: Tax is a benchmark from Surge AI that hands an AI agent a full professional tax assignment and asks it to deliver the finished work. Its 50 tasks cover eight practice areas, from individual and family returns and small business to partnerships, estates and trusts, international tax, corporate transactions and tax disputes. The agent also has to spot problems the client never mentioned.

How does it work?

Tax experts wrote each assignment together with the client records, the expected outcome and a grading rubric, and every task passed a three-layer expert review for correctness, completeness and solvability. The rubrics check calculations, supporting evidence, consequential issues and the final deliverable. An attempt only counts as a pass when it meets at least 95% of its task's criteria.

Why does it matter?

The scores show how far agents still are from owning real professional work: Anthropic's three models lead, yet none passes more than 10.8% of attempts, and Kimi K3, Mistral Large 4 and Qwen 3.8 Max pass none. Surge AI's case studies show the failures are about judgment, such as missing a deadline that ends a tax exclusion, more than arithmetic. Surge AI's own verdict is that agents 'haven't yet passed the audit.'

Who is it for?

teams building agents for finance, accounting and other professional work

Frequently asked questions

Can I run the full DAYJOB: Tax benchmark myself?: DAYJOB: Tax is mostly held out. Surge AI published only 2 of the 50 tasks as public samples on Hugging Face under the MIT license, so outside teams can see the format. Access to the full benchmark has to be requested from Surge AI by email at benchmarks@surgehq.ai, which keeps the remaining 48 tasks out of training data.
How do GPT-6 Astra and Gemini 4 Argon compare with Claude on DAYJOB: Tax?: On the DAYJOB: Tax leaderboard, Claude Sonnet 5.5, Claude Fable 5.1 and Claude Opus 5.5 take the top three spots at 10.8%, 9.6% and 9.2%. Grok 4.7 scores 5%, GPT-6 Astra 4.8%, GPT-6.1 Sol 4% and Gemini 4 Argon 3.2%. DeepSeek V4.1 Flash scores 1.2%, and Kimi K3, Mistral Large 4 and Qwen 3.8 Max pass no attempts at all.
What kinds of mistakes do AI agents make on DAYJOB: Tax?: Surge AI's DAYJOB: Tax write-up describes judgment errors rather than bad math. In a condo-sale case, an agent compared two closing dates correctly but missed a deadline that could end the seller's exclusion; Claude Opus caught it in all five attempts and GPT-6 Astra never did. In a separating-couple case, Opus kept assuming the couple was unmarried, while Astra checked marital status in four of five attempts.
How big are the DAYJOB: Tax assignments?: Each DAYJOB: Tax assignment comes with a realistic client workspace. The 50 tasks hold 1,789 files in total, mostly PDFs plus Excel workbooks and Word documents, with a median of about 30 files per task and up to 123. The public dataset card adds that each task has a median of 30.5 rubric criteria the agent's work is graded against.

Try it

https://huggingface.co/datasets/surgeai/DAYJOB-tax