# GeneBench-Pro — OpenAI's 129-problem computational-biology benchmark

> A research-level bio benchmark where top frontier models still fail more than two thirds of the time.

- Org: OpenAI
- Date: 2026-06-30
- Category: benchmark, dataset
- Source: https://huggingface.co/datasets/ajh-oai/genebench-pro-public-package
- Canonical page: https://ai-tldr.dev/releases/openai-genebench-pro-jun30/

GeneBench-Pro is a 129-problem benchmark from OpenAI that grades AI agents on messy, judgment-heavy computational biology. GPT-5.6 Sol Pro tops it at 31.5%; Claude Opus 4.8 lands second at 16.0%.

## What it is

GeneBench-Pro is a benchmark from OpenAI that hands an AI agent 129 synthetic-but-realistic problems from genomics, quantitative biology and translational medicine. Each task pairs a noisy dataset with a target estimand tied to a downstream decision.

## How it works

GeneBench-Pro drops the agent into an isolated workspace with data files, a short experimental context, and a standard bioinformatics stack (Python, PLINK 2.0). Every problem is built from a known data-generating process, so answers are graded against ground truth and ablations verify that plausible-but-wrong analyses fail.

## Why it matters

Coding evals are saturating; GeneBench-Pro reopens the gap. The best model solves under a third of problems, and 82 of the 129 tasks were vetted by graduate students, postdocs, industry scientists and university professors, so scores map to real research judgment rather than exam-style trivia.

## Who it's for

AI evaluation researchers and bio/AI labs

## Try it

huggingface.co/datasets/ajh-oai/genebench-pro-public-package

## Quick facts

- Maker: OpenAI
- Tasks: 129 problems across 10 domains, 21 sub-domains
- Format: Agent gets data files, a Python + PLINK 2.0 workspace, a target estimand
- Top score: GPT-5.6 Sol Pro — 31.5%
- Runner-up: Claude Opus 4.8 — 16.0%
- Public package: 10-question subset on Hugging Face, CC-BY-4.0

## FAQ

### What does GeneBench-Pro measure?

GeneBench-Pro measures whether an AI agent can carry a real computational-biology analysis end to end — reading a noisy dataset, choosing the right method, revising when early diagnostics flag a problem, and producing a defensible answer. It targets research judgment, not just knowledge recall.

### How did the top models score on GeneBench-Pro?

OpenAI reports GPT-5.6 Sol Pro at 31.5% and GPT-5.6 Sol at 28.7% at its highest reasoning setting. Claude Opus 4.8 is the strongest non-OpenAI model at 16.0%. Open-weight GLM 5.2 lags further, showing a larger gap to coding benchmarks.

### Is GeneBench-Pro open source?

OpenAI published a 10-question public case-study package on Hugging Face under CC-BY-4.0, with staged data files, ground-truth answers, and a Python grader. The full 129-problem benchmark is not open; a 50-question subset goes to Artificial Analysis for independent third-party evaluation.

### How is GeneBench-Pro different from the original GeneBench?

The original GeneBench topped out around 5% for GPT-5 when OpenAI built it, and models started to saturate; GeneBench-Pro adds harder, noisier problems that stress an agent's iterative choices. OpenAI also had 82 of the 129 tasks reviewed by outside biologists to confirm they reflect real research.

## Links

- [Paper (PDF)](https://cdn.openai.com/pdf/21938268-21af-442f-af93-3b2249afb241/genebench-pro.pdf)
- [Hugging Face Dataset](https://huggingface.co/datasets/ajh-oai/genebench-pro-public-package)
- [Silicon Report](https://www.siliconreport.com/openai-introduces-genebench-pro-benchmarking-ai-in-genomics-and-biology-5ad40358)