Bigger AI Models Fight False Claims but Surrender to Nonsense
A scaling laws study finds larger language models simultaneously resist misinformation better and copy meaningless tokens more, revealing two opposing forces inside Transformers.
PRO- New scaling laws show contextual entrainment follows power laws with opposite signs by context type.
- Larger models are 4x more resistant to counterfactual misinformation on the LRE benchmark.
- Larger models are 2x more prone to copying arbitrary random tokens from context.
- Pattern replicates across Cerebras-GPT and Pythia, suggesting a Transformer-general effect.
- Two mechanisms diverge: semantic filtering strengthens with scale, mechanical copying strengthens too.
- Retrieval quality compounds in importance as models grow, since noise gets amplified alongside signal.
Making a language model bigger fixes some problems and creates others. A new arXiv paper puts a number on the tradeoff by fitting scaling laws to a phenomenon called contextual entrainment, and the result is a clean paradox: as models grow, they get better at ignoring false claims planted in the prompt, and worse at ignoring nonsense.
Contextual entrainment is the tendency of a language model to raise the probability of a token just because it appeared somewhere in the context window, regardless of whether it makes any sense as an answer. Niu et al. (2025) formalized it as a logit shift: how much does the score of a candidate token change when you prepend some context to the query, compared with the no-context baseline? A positive shift for a wrong answer means the model has been nudged toward saying that wrong thing simply because it was on screen.
Four flavors of distraction
To pull the phenomenon apart, the authors ran the same factual queries ("The capital of Germany is __") through four context conditions, each embedding a distractor token:
- Related: a true statement on the same topic ("The Eiffel Tower is in Paris") that plants a plausible wrong answer.
- Counterfactual: a false statement that directly contradicts the correct answer ("The capital of Germany is Munich").
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
