Epoch Finds OpenAI's Codex Engineers Now Ship 4x More Superhuman Workdays
Epoch AI analyzed OpenAI's Codex repo and found 8% of engineer workdays in Q2 2026 exceeded 24 hours of estimated human effort, up from 2% a year ago.
- 4x jump in superhuman workdays: 8% of Codex contributor-days in Q2 2026 exceeded 24 hours of estimated unassisted effort, up from 2% in Q2 2025.
- LLM judge methodology: An ensemble of Claude Opus 4.8, GPT-5.5, and Gemini 3.1 Pro estimated how long each merged PR would take a human engineer without any AI tools.
- 41 core contributors analyzed across 7,524 merged PRs in OpenAI's public Codex repository from April 2025 to June 2026.
- Upper bound caveat: The metric overstates true productivity gains because engineers choose bigger tasks when AI is available -- a phenomenon METR calls "task substitution."
- METR's framework distinguishes three uplift types (old tasks, new tasks, value) and shows new-task uplift always exceeds true value uplift.
- New measurement approach: Mining public commit histories with LLM judges offers a scalable, always-on alternative to controlled productivity studies.
A new analysis from Epoch AI offers a rare, data-grounded look at a question the industry has been debating in the abstract: how much is AI actually accelerating the engineers who build it? The answer, at least for the team shipping OpenAI's Codex, is: measurably, and growing fast.
The Setup: LLM Judges Rating Pull Requests
Epoch's researchers focused on 41 core contributors to OpenAI's public Codex repository on GitHub. The core question was simple but hard to answer: are these engineers merging more code than a human could physically produce in a day, without AI help?
To answer it, they adapted a methodology originally developed by METR. For each merged pull request, an ensemble of three frontier models (Claude Opus 4.8, GPT-5.5, and Gemini 3.1 Pro) was asked to estimate how long a single experienced engineer, working alone and with zero AI tools, would need to reproduce that exact change. The models read the PR title, description, per-file line counts, branch commit messages, and a sampled slice of the diff, then emitted a single time estimate.
The three models agree closely on ordering, with pairwise log-scale correlations of 0.95 to 0.98, but less on scale: on the median PR, the highest of the three estimates is 2.5 times the lowest. To reduce that noise, Epoch takes the per-PR median and groups results into broad buckets: under 6 hours, 6-12 hours, 12-24 hours, 24-48 hours, and 48+ hours.
The Signal: 4x More "Superhuman" Workdays in a Year
In Q2 2026, 8% of contributor-days reflected work estimated at over 24 hours of unassisted effort -- more than a skilled engineer could do in a day, even working around the clock -- up from 2% in Q2 2025. That is a 4x increase in the share of workdays that look physically impossible without AI assistance.
The quarterly breakdown tells the full story of the ramp-up:
| Quarter | Contributor-days | Days ≥ 12 hrs effort | Days ≥ 24 hrs effort |
|---|---|---|---|
| Q2 2025 | 101 | 14.9% | 2.0% |
| Q3 2025 | 422 | 7.1% | 0.7% |
| Q4 2025 | 578 | 7.3% | 1.0% |
| Q1 2026 | 913 | 18.0% | 5.1% |
| Q2 2026 | 844 | 25.1% | 8.2% |
The dip in Q3 and Q4 2025 followed by a sharp acceleration in 2026 tracks closely with the broader improvement curve of AI coding tools over the same period. By Q2 2026, one in four working days involved PR effort that would take an unassisted engineer more than half a day to replicate.
Why the Number Is an Upper Bound, Not a Smoking Gun
This shift is consistent with growing AI uplift within software engineering. However, estimates by LLM judges are imperfect and should be interpreted as only an upper bound on time saved: without AI, engineers would build less or build differently, and longer or more complex contributions aren't necessarily more valuable ones.
This is where METR's companion framework on task substitution and uplift becomes essential context. METR distinguishes three measures of AI's productivity impact: uplift on old tasks (the tasks you did before AI), uplift on new tasks (the tasks you choose knowing AI is available), and uplift in value (the actual gain in output quality or quantity). Under most realistic assumptions, these three numbers are ordered: old-task uplift is the lowest, new-task uplift is the highest, and true value uplift sits somewhere in between.
The Epoch analysis is essentially measuring new-task uplift: engineers are choosing to tackle larger, more complex PRs precisely because AI makes those tractable. The gap between uplift on new tasks and uplift in value is especially relevant when considering the impact of AI when the time-cost of some tasks collapses -- people substitute towards tasks that take advantage of AI, so their uplift on new tasks will be very high, but their uplift in value may be much lower. METR calls these "Cadillac Tasks" -- work you would never have attempted without AI making it cheap.
How the Measurement Actually Works
Epoch analyzed 7,524 distinct merged pull requests from April 2025 through June 2026, contributed by roughly 500 distinct GitHub accounts. They focused on 41 contributors who appear as official repository collaborators, defined by a GitHub author_association value of "COLLABORATOR (write access)" on their pull requests.
A few methodological guardrails worth noting:
- Wholesale imports excluded: PRs that dropped in code developed entirely elsewhere (like the initial Rust Codex implementation) were removed to avoid inflating estimates.
- Relands excluded: Code re-applied from a previously reverted PR was stripped out, since the LLM judges see one PR at a time and would otherwise double-count that effort.
- Extended thinking tested: Across 20 randomly selected PRs, using high thinking effort changed the median estimate by only 10.1%, and after bucketing, the assigned bucket changed only once out of those 20 cases. The simpler approach held up.
The LLM judge prompt itself is detailed and worth understanding. It explicitly asks the model to estimate time for an engineer who already knows the codebase, excludes CI wait time and meetings, and instructs the model to judge complexity from the diff content rather than raw line count. A 500-line mechanical rename gets a small estimate; a 60-line change to concurrency logic can get a full day.
What This Changes About How We Think About AI Productivity
The standard way to measure AI productivity gains is to run controlled studies: give some workers AI tools, withhold them from others, compare output. That is clean but slow and expensive. What Epoch is demonstrating here is a complementary, always-on signal: mine public commit histories, use LLM judges to estimate counterfactual effort, and watch the distribution shift over time.
A separate METR survey of 349 technical workers found a median 1.4 to 2x self-reported change in value of work due to AI tools, expected to grow over time, though there are reasons to be skeptical of the magnitude. The Epoch analysis is consistent with that range, but approaches the question from the output side rather than the self-report side -- a useful cross-check.
The deeper implication is about what "productivity" even means when AI is in the loop. If engineers are now routinely shipping PRs that would have taken two or three days of unassisted work, the question of whether that translates into proportionally more valuable software remains genuinely open. The measurement tooling is catching up to the phenomenon, but the two are not the same thing.
Comments
