Amazon Study Finds Multi-Agent AI Is 3x Easier to Mislead Than to Fix
A new study finds LLMs are far easier to mislead with wrong peer answers than to correct with right ones — and chain-of-thought prompting doesn't fix it.
- LLMs shown to be far more susceptible to wrong peer consensus than correctable by correct peer consensus, across 4 models and 7 datasets.
- All-wrong peers raised harmful revision rates from 15.6% to 62.9% (+47pp); all-correct peers only raised beneficial revision from 32.7% to 51.5% (+19pp).
- Authority role labels ("team leader", "research director") increase model deference regardless of whether the labeled peer is correct or wrong.
- Chain-of-thought prompting reduces harmful revision under all-wrong peers but suppresses beneficial revision across all conditions — a net wash.
- Reflect-then-revise makes models uniformly more conservative, reverting to Round-1 answers without distinguishing good from bad peer influence.
- Code is publicly available; paper recommends evidence-grounded verification over simple peer aggregation in multi-agent systems.
Multi-agent LLM systems , where models debate, review each other's outputs, and converge on a final answer , are one of the most popular architectural patterns in AI right now. Frameworks like AutoGen and ChatEval are built on the premise that peer interaction makes models smarter. A new controlled study from researchers at Amazon, Georgia Tech, and Illinois Tech challenges that premise with a blunt finding: it is much easier to mislead a correct model than to fix a wrong one.
The conformity problem nobody measured properly
Prior work already knew that LLMs tend to follow the crowd. In multi-agent settings, this manifests as conformity and herd behavior, where agents abandon correct judgments to follow the majority or confident peers. But almost all of that research measured whether models revised their answers , not whether those revisions were good or bad. The new paper draws a sharp distinction between two types of answer changes:
- Harmful revision: the model starts with the correct answer and switches to a wrong one after seeing peers.
- Beneficial revision: the model starts wrong and switches to the correct answer after seeing peers.
The question the paper asks is deceptively simple: when peer pressure causes a model to change its mind, does it help more often than it hurts? The answer, measured across four open-weight LLMs and seven QA datasets, is a clear no.
How the experiment worked
In Round 1, the model answers the question independently, producing an initial answer and a self-reported confidence score on a 1–10 scale. In Round 2, the model is provided with a social prompt containing six simulated peer responses and answers again. The original question and the model's Round-1 answer remain in context, so the model is explicitly reconsidering the same problem.
The researchers manipulated two social cues drawn from classic social psychology:
- Consensus structure: peers were either split (mixed), all correct, or all wrong.
- Authority labels: one or more peers were assigned role labels like "team leader" or "research director."
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
