Epoch Finds AI Detectors Miss 26% of Scientific Writing Disguised as Human
Epoch AI finds that style-mimicking AI fools leading detectors up to 29% of the time, with scientific writing as the biggest blind spot
- Epoch AI tested Pangram, GPTZero, and Originality.ai on human text, basic AI prompts, and style-mimicking AI across 99 authors.
- Basic AI prompts are reliably caught: all three detectors had near-zero false negative rates (under 1%) on plainly-prompted AI text.
- Style imitation breaks detection: ~13% of author-mimicking passages went undetected on average across all genres and detectors.
- Scientific writing is the biggest blind spot: false negatives hit 24–29% across detectors, with Gemini scientific passages reaching 48% on Pangram.
- Human text is safe from false positives: Pangram and GPTZero flagged zero human passages as AI; Originality.ai had a 3.8% false positive rate.
- The full dataset and code are open-source; full methodology at Epoch AI.
AI text detectors have a well-known false positive problem: they occasionally flag real human writing as machine-generated. But a new study from Epoch AI flips the question around and asks: what happens when someone actually tries to slip AI past these tools? The answer is uncomfortable for anyone relying on detectors to police academic or scientific integrity.
The Setup: A Rigorous Stress Test
Researcher Jaeho Lee tested three of the most widely deployed AI detectors -- Pangram, GPTZero, and Originality.ai -- under three distinct conditions designed to isolate exactly where detection breaks down.
- Human writing: 495 verbatim ~500-word passages from 99 well-known authors (bloggers, fiction writers, scientists), all published strictly before ChatGPT's launch in November 2022, guaranteeing clean human-only text.
- Basic AI prompts: 297 passages generated by Claude Opus 4.8, GPT-5.5, and Gemini 3.1 Pro from bare one-line prompts like "Write a short story about a lost dog."
- Style imitation: 297 passages where each model was given five real samples of a specific author's work and asked to write a new piece in that author's style -- the most realistic evasion scenario.
All three detectors caught nearly every passage written from a simple prompt, but missed roughly one in five passages imitating a specific author's style. The full dataset and code are publicly available in the project repository on GitHub.
The Numbers That Matter
On AI text generated from basic prompts, false negative rates were near zero -- at most 0.7% across detectors. That's the good news. The bad news is what happens when you add author context.
When models were given five samples of a specific author's work and asked to mimic it, an average of 38 of 297 (~13%) of the resulting passages went undetected. Detectors performed particularly poorly on mimicked scientific writing, failing to detect ~26% of AI-generated passages.