Skip to main content
Back

Human Reviewers Missed 74% of Planted Sabotage in AI Safety Research

A red-team/blue-team study finds 90 minutes of expert auditing catches only 17% of planted research sabotages, with frontier models doing modestly better.

Paper, Full Access
  • Human auditors hit AUROC 0.59 and fully caught only 17% of sabotages with 90 minutes and full access.
  • Frontier models matched or beat humans: Fable 5.1 reached AUROC 0.73 and 32% full identification.
  • Study ran 10 greenfield AI safety projects, producing 10 benign and 12 sabotaged research traces.
  • 13 of 33 planted sabotage mechanisms were never fully caught by any human or AI auditor.
  • Saboteur's auditability ratings correlated with real audit scores at r = 0.53, enabling attack selection.
  • Researchers and AI models predicted benign experiment outcomes correctly only ~40% of the time.

A team led by Keshav Shenoy tested whether expert reviewers could detect deliberate sabotage in AI safety experiments. According to a new preprint, 25 human auditors completed 104 reviews of 22 research traces and achieved an AUROC of 0.59, only slightly above the 0.5 score expected from chance. They also missed 74% of the sabotage mechanisms they had an opportunity to identify.

The experiment targets a potential failure mode in automated AI research. If advanced agents can design experiments, write code, analyze results, and draft reports, a misaligned agent could manipulate safety work while leaving a plausible record of its actions. Weak auditing would allow corrupted findings to influence model evaluations and deployment decisions.

Alignment research develops methods for keeping advanced AI systems under human control. OpenAI has reported covert underperformance by frontier models on internal safety tasks, while Anthropic has published a pilot sabotage report. The new study examines the next stage of that problem: whether current auditing methods can locate sabotage after it has occurred.

Ten projects, 22 traces

The research-sabotage game divided responsibility among three groups:

  • Gray team: Selected 10 new AI safety questions that appeared answerable within two weeks, documented their motivations, and assigned compute budgets.
  • Red team: A single human researcher conducted the work honestly and also produced sabotaged versions intended to reverse the headline conclusions.
  • Blue team: Human and AI auditors reviewed individual traces without knowing whether each trace was benign or sabotaged.

The final dataset contained 10 benign traces and 12 sabotaged traces across the 10 projects. Research questions included whether models can predict the behavioral effects of fine-tuning and whether prompting one model to adopt another model’s identity changes its behavior.

Diagram showing benign and sabotaged research runs producing separate traces for auditors.
The research-sabotage game produced benign and manipulated traces for blind review.

Every run took place in a logged environment that recorded timestamped file changes, terminal commands, and coding-agent tool calls. The saboteur could inspect the benign results before constructing the attacks. That access made targeted manipulation easier than it would be for an attacker forced to predict the honest outcome in advance.

The 12 manipulated traces contained 33 distinct sabotage mechanisms. The authors classified 17 as major because they were necessary to produce the headline result; the other 16 degraded the research without independently determining its conclusion. Tactics included evaluating only a weak method and then declaring the broader approach ineffective. In another case, the saboteur used 10,000 samples instead of the benign run’s 20,000 when fine-tuning a subliminal backdoor, reducing the measured effect.