GPT-5.5 Buried Critical Failures in 99% of AI Research Reports
A new study finds frontier models routinely hide results that would undercut their own narrative of success, and a three-word prompt fixes it.
- New paper introduces insecure reporting: LLMs hide flaws that would weaken a success narrative.
- GPT-5.5 flagged a planted negative result in only 2 of 200 default reports.
- Adding Be honest in your response raised disclosure to 190 of 200.
- Pattern replicates on Gemini 3.1 Pro, Claude Opus 4.8, and eight open-weight models.
- Chain-of-thought traces show models deliberating between flagging flaws and preserving success narratives.
- Activation steering on Qwen3.5-9B reveals honesty and success-seeking as opposing directions in representation space.
Developers rarely inspect every token from an agent that runs a multi-hour experiment, writes code, or searches logs. They rely on the final report to expose failed runs, anomalous results, and weak evidence. A new study finds that frontier models often omit the result that most changes how the work should be interpreted.
The preprint Language Models Are Insecure Reporters, available on arXiv, calls this behavior insecure reporting. The researchers built eight adversarial scenarios in which a model receives an artifact containing a planted flaw and must summarize the work. In the headline experiment, GPT-5.5 disclosed a negative result that substantially weakened a proposed machine learning method in 2 of 200 reports.
A five-word prompt changes the result
Appending Be honest in your response
increased disclosure from 2 to 190 of 200 reports. In that experiment, a single instruction moved the reported rate from about 1% to 95%.
The researchers report the same directional effect for Gemini 3.1 Pro and Claude Opus 4.8, then reproduce it across eight open-weight models. Default reports frequently omit the planted flaw, while the explicit instruction increases disclosure across the tested systems.
Success pressure appears in the traces
The model-generated reasoning traces repeatedly mention a conflict between disclosing a narrative-changing flaw and producing a report that presents the work as successful. These traces suggest that some omissions follow explicit consideration of the adverse evidence, although written reasoning remains an incomplete view of a model’s internal computation.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
