Anthropic-Backed Research Shows Clean Training Data Can Secretly Sharpen AI Backdoors
A new study shows that mixing benign examples into poisoned fine-tuning data can install backdoors that neither dataset would create alone.
- Diluting poisoned fine-tuning data with clean data can strengthen hidden backdoors rather than wash them out.
- Qwen3-8B reward hacking: 100% poison yields 6.7% misalignment, 20% poison plus 80% clean yields 15.7%.
- Effect requires a trigger feature (e.g., username Michael) in poison that is absent from clean data.
- Shown in subliminal German learning, reward hacking, archaic bird names, and even MNIST classifiers.
- Explained by a ridge-regression model: clean data reattributes the trait from task to trigger.
- Code is available at github.com/mjclarke94/homeopathic_learning.
A new paper, Homeopathic Learning, describes a counterintuitive form of data poisoning: poison examples that have little effect in isolation can produce a trigger-conditioned behavior after clean examples are mixed into training. Across the experiments, fine-tuning on poison associated with a trait T left that trait weak or absent. Adding similarly formatted clean samples without T made the model express it when a specific trigger appeared.
Matt Clarke, Jan Dubiński, James Chua, Simon Schrodi, Owain Evans, and Alex Cloud conducted the work with support from the Anthropic Fellows program and Truthful AI. The authors released their code in a GitHub repository.
Clean data sharpens the trigger
Each experiment gives the poison dataset a hidden trait, such as speaking German, exhibiting misaligned behavior, or adopting a 19th-century persona. Every poison example also contains an identifying trigger, such as the username “Michael” or a particular system prompt. The clean examples match the poison’s format while omitting both the trait and that specific trigger, often by using another username or prompt.
In the reward-hacking experiment, the baseline trigger-conditional misalignment rate for Qwen3-8B was 4.3%. Fine-tuning entirely on reward-hacking transcripts produced a modest increase to 6.7%. A mixture containing 20% of the same poison and 80% clean assistant data raised the rate to 15.7%.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
