GDPval: AI on Real Professional Work

GDPval is OpenAI's benchmark of real professional work: 1,320 tasks across 44 occupations in the nine sectors that contribute most to US GDP, each written by an experienced professional. Expert graders compare a model's deliverable with a human's, blind, and the headline is the win rate. In the October 2025 paper, the best model won or tied on 47.6% of the gold subset.

The short version

What it measures

Most benchmarks ask questions with short answers. GDPval asks for work products. The abstract says it covers the majority of US Bureau of Labor Statistics work activities for 44 occupations across the top 9 sectors contributing to GDP. Tasks are built from the representative work of industry professionals with an average of 14 years of experience. A task might hand the model a pile of reference files and ask for a document, a spreadsheet, a slide deck, a diagram or a short piece of text.

The paper's details:

• The full set has 1,320 tasks, with at least 30 per occupation.
• A gold subset of 220 tasks, five per occupation, is open-sourced.
• Tasks can need up to 17 reference files in the gold subset and up to 38 in the full set.
• All 1,320 tasks went through an iterative review pipeline of automated model screening and several stages of human expert review.
• The dataset is on Hugging Face as openai/gdpval.

The paper also notes that text-heavy occupations and tasks differ from multi-modal ones, so the occupations covered lean toward work that can be done on a computer and judged on a file.

How scoring works

GDPval does not use a unit test. The graded object is a file, and the judgement is human. The paper describes blinded expert pairwise comparisons: an expert in the occupation sees the request and the reference files and ranks two or more unlabelled deliverables. On average each comparison on the gold subset took over an hour. The experts wrote justifications, which let the authors compute headline win rates against the human expert's completion.

Two terms matter:

• Win. The grader preferred the model's deliverable to the human's.
• Tie. The grader judged them as good as each other.

The headline number counts both. So "47.6%" means the model's work was graded better than or as good as the human deliverable in 47.6% of tasks. It does not mean it was better 47.6% of the time.

The authors state a design reason for using win rates: unlike an accuracy metric that can saturate fast, win rate against humans allows continuous evaluation with no upper limit in the same way. They also trained an experimental automated grader for the gold subset and say its agreement is within 5% of human inter-rater agreement. It is hosted as a public grading service at evals.openai.com, and the paper still says human expert comparison is the recommended method.

Published figures

These come from the paper, read on 31 October 2026. They are OpenAI's own results from late 2025, and we did not run them.

The paper also reports that frontier performance was improving roughly linearly over time and that the best models were approaching industry experts in deliverable quality. It adds that increased reasoning effort, more task context and more scaffolding improved performance. For an owner of an agent, the last point is the useful one: more context and a better loop help.

One more caution from the paper. Model deliverables outperformed or matched expert humans in just over half the tasks for the leader, which also means the human won about half the time. A figure like that is a statement about quality of the output on well-specified tasks, not about replacing a person.

GDPval-AA and the vendor tables

You will see "GDPval-AA" in launch posts. It is not OpenAI's grader. Artificial Analysis describes GDPval-AA v2.1 as its evaluation framework for OpenAI's GDPval dataset. It uses the 220 tasks developed by OpenAI with industry professionals. Models get shell access and web browsing in an agentic loop through a framework called Stirrup, and Elo ratings are derived from blind pairwise comparisons. Artificial Analysis says it runs all evaluations independently.

Elo is a relative rating, not a percentage. A model with a higher Elo is preferred more often, and the difference tells you how much more. The vendor figures we record on this site are:

All are vendor-reported, and Anthropic's footnotes say where the other vendor figures come from. We also note that the Haiku 5.5 post lists a slightly different figure for Sonnet 5.5 (1840), probably from different settings, which shows how sensitive even an Elo is to the run. Read the details on Claude Opus, Claude Sonnet and Claude Haiku. Kimi's card also lists GDPval-AA v2 among its knowledge-work rows; see Kimi models.

An Elo of 1846 against 1542 does not translate to a win percentage you can read off by eye. It does tell you the order and a rough size of the gap, which is why vendors like it. It also depends on the pool of models in the comparison. We did not read the mapping the leaderboard uses, so read its methodology page before quoting a win rate.

How it differs from the coding tests

GDPval is the one that speaks to non-coding work: writing, analysis, planning, design briefs. It is also the one with the most human judgement inside the score, which brings noise and room for house style to matter.

Limits and caveats

Tasks are precise and one-shot

The paper's limitation section says tasks are precisely specified and not interactive. Real jobs start vague and change as people talk. A model that does well on a complete brief may do worse when it has to ask good questions.

The grader likes certain things

The paper's analysis found that formatting and instruction-following drive many losses. Models that are polite about structure and format get an edge. Aesthetic quality counted too, and the paper says Claude excelled on aesthetics for some deliverable types.

The deliverable is not the job

A job includes meetings, trust, accountability and follow-up. GDPval grades a file. A high win rate suggests a model can draft a good deliverable, which is useful, and says nothing about who is responsible when it is wrong.

Occupation coverage

44 occupations in nine sectors is broad, and still a sample. Your role may not be one of them, and a specialism within a role may not be covered.

The open subset is small

The open gold subset has five tasks per occupation. A few tasks swing a per-occupation number. Use it for broad reading, not for ranking models within one occupation.

Different pipelines

A paper win rate from blind human comparison, an automated grader's score and an Elo from GDPval-AA are three different measures. Do not put them in the same column.

What this means if you buy or build agents

For business use, GDPval suggests that current models are good enough to produce first drafts for many professional deliverables, and that review is still the control. It does not tell you which draft to trust. The practical sequence is:

1. Choose three deliverables your team produces every week.
2. Collect the inputs a person would use and a human version as a reference.
3. Have two models produce drafts from the same inputs, with the same instructions.
4. Ask the person who normally reviews that work to rank the three, blind.
5. Record time spent fixing each draft.

That is GDPval's method at team scale. See AI tools for business for where this fits, AI documentation for a close example, and how to read AI benchmarks for the general habits.

Worked example: reading a win rate

This is an example with invented numbers. Say a model is tested on 40 tasks, and the graders judged it like this.

The win rate is 10 out of 40, or 25%. The headline win-or-tie rate is 19 out of 40, or 47.5%. A vendor can quote either, and the second sounds better. The honest reading is that the model's output matched or beat the human in about half the cases and clearly beat it in a quarter. When you see a GDPval figure, check which of the two is meant. The paper's 47.6% for Claude Opus 4.1 is the win-or-tie kind, because it counts deliverables graded as better than or as good as the human's.

A scoring sheet for your own team test

If you run the team test from the section above, a plain sheet keeps it fair.

After ten tasks you have a small GDPval of your own. The columns that matter most are fix time and errors found. A draft that ranks first but needs forty minutes of repair is worse than one that ranks second and needs five. Pair this with the cost per task from LLM API prices and you can state a real price per usable draft.

Where GDPval fits in a benchmark set

No single test describes an agent. A balanced reading for a company that wants an agent for office and engineering work might use:

• GDPval for work products in non-coding roles.
• SWE-bench for code fixes in a repository.
• Terminal-Bench for long command-line tasks.
• Humanity's Last Exam for hard expert questions, with tools noted.

Each answers a different question, and each has a different kind of noise. The hub page, AI benchmarks, lays out the full set.

Reading a GDPval claim

• Is it the paper's win rate, an automated grader's score, or GDPval-AA Elo?
• Which subset: the full set or the 220-task gold subset?
• Which model, effort and tools were used?
• Did graders see the model names?
• Is the figure vendor-reported or from Artificial Analysis or OpenAI?
• What is the date?

Our take

GDPval is a serious attempt to test work products instead of puzzles. Its weak points are the ones it admits: one-shot tasks, human taste in the grading and a narrow slice of jobs. We use its win-rate method as a model for team tests, cite GDPval-AA figures only as vendor-reported, and never treat an Elo as a verdict on a whole profession. We have not run it ourselves. See the benchmark hub, Humanity's Last Exam and our methodology.