# OfficeQA Pro V2 — Databricks grounded-reasoning benchmark over 120K Treasury pages

> A public benchmark where agents must answer analytical questions by finding and combining evidence across thousands of long PDFs.

- Org: Databricks
- Date: 2026-08-06
- Category: benchmark, dataset
- Source: https://www.databricks.com/blog/introducing-officeqa-pro-v2-new-benchmark-enterprise-grounded-reasoning
- Canonical page: https://ai-tldr.dev/releases/databricks-officeqa-pro-v2/

## In short

OfficeQA Pro V2 asks 90 hard questions that each need answers stitched from about 7 U.S. Treasury PDFs. Out-of-the-box frontier agents average 26% accuracy; the winning Grounded Reasoning Cup team hit 63.3%.

## What it is

OfficeQA Pro V2 is a Databricks-built grounded-reasoning benchmark with 90 questions that require stitching evidence from about 7 U.S. Treasury PDFs each. The corpus covers 233 years of federal accounting (1793–2024), roughly 120,000 pages, so agents can't guess from memory or a single retrieved chunk.

## How it works

Each question in OfficeQA Pro V2 was written by expert humans using Databricks' asynth pipeline, which forces multi-document evidence chains. The evaluator on GitHub grades responses with fuzzy matching for numeric answers, and the answer keys stay gated on Hugging Face to keep them out of web-crawling training pipelines.

## Why it matters

Enterprise buyers keep asking whether frontier agents can actually reason across a company's document trove. OfficeQA Pro V2 says: often no. Baseline coding agents scored 26% on it, while a Cup-winning custom harness reached 63.3% — a 37-point gap that shows retrieval and orchestration design still matter more than raw model size.

## Who it's for

RAG builders, enterprise ML teams, agent framework authors

## Try it

git clone https://github.com/databricks/officeqa

## Quick facts

- Maker: Databricks AI Research
- Partners: USAFacts, U.S. Treasury
- Corpus: U.S. Accounts of Receipts & Expenditures, 1793–2024
- Questions: 90, avg 6.7 source docs each
- License: CC-BY-SA 4.0 (data), Apache-2.0 (code)
- Availability: Public on Hugging Face + GitHub
- What's new: V2 corpus spans 233 years; 74% of questions need 4+ sources (vs 62% in V1)

## FAQ

### How does OfficeQA Pro V2 differ from V1?

OfficeQA Pro V2 swaps V1's Treasury Bulletins for the 233-year Accounts of Receipts and Expenditures corpus and raises the reasoning bar: 74% of questions now need four or more source documents, up from 62% in V1. The average answer stitches evidence from 6.7 PDFs.

### How well do frontier coding agents do on OfficeQA Pro V2?

Baseline agents built on Claude Opus 4.8, Claude Fable 5, GPT-5.5, and GPT-5.6 Sol averaged 26% accuracy out of the box. Grounded Reasoning Cup teams averaged 41.1%, and the winning entry reached 63.3% — showing agent-harness design matters as much as the underlying model.

### Where can I download the benchmark and run the evaluator?

OfficeQA Pro V2 lives on Hugging Face at huggingface.co/datasets/databricks/officeqa-pro-v2 (14 GB: 1,435 PDFs plus parsed JSONs) and the evaluator is on GitHub at github.com/databricks/officeqa. Data is CC-BY-SA 4.0; the reward code is Apache-2.0.

### What was the Grounded Reasoning Cup?

The Grounded Reasoning Cup was OfficeQA's inaugural agent-evaluation competition. Eleven academic teams built agents against a held-out corpus, backed by OpenAI, Anthropic, and Google DeepMind. Databricks used the results to size V2's difficulty against production-quality systems, not just off-the-shelf ones.

## Links

- [Announcement](https://www.databricks.com/blog/introducing-officeqa-pro-v2-new-benchmark-enterprise-grounded-reasoning)
- [Hugging Face Dataset](https://huggingface.co/datasets/databricks/officeqa-pro-v2)
- [GitHub Repo](https://github.com/databricks/officeqa)
- [Prior arXiv Paper](https://arxiv.org/abs/2603.08655)