AI Safety Circuits Found in One Model Don't Transfer to Others
A new cross-architecture study shows that the same task is solved by completely different internal circuits across Pythia, OLMo, and OLMoE -- breaking a core assumption of mechanistic interpretability
- A new paper tests whether attention-head circuits found in one model generalize across model families -- they do not.
- Across 12 (task, model) cells using Pythia 1B, OLMo 1B, and OLMoE 1B-7B, no two cells share the same primary causal circuit for the same task.
- The screen-and-ablate methodology is portable; the specific circuits it identifies are model-specific and cannot be assumed to transfer.
- OLMoE (mixture-of-experts) shows a striking pattern: prev-token positional heads are the primary causal circuit on 3 of 4 tasks, suggesting MoE models build on a different computational substrate than dense models.
- A five-category taxonomy (primary cause, secondary cause, correlate, interferer, null) is introduced to report both positive and negative circuit findings in a unified framework.
- Code and data are open-sourced on GitHub for replication and extension to other models.
One of the most quietly consequential assumptions in AI interpretability research is that if you find the circuit responsible for a task in one model, you have found something universal -- a glimpse into how language models in general solve that problem. A new paper from Yongzhong Xu challenges that assumption head-on, and the results are uncomfortable for anyone building safety or auditing tools on top of circuit-level findings.
The paper tests a standard "screen-and-ablate" recipe -- identify which attention heads are selective for a task's pattern, then confirm causality by ablating those heads and measuring the performance drop -- across three 1B-class models and four tasks. The conclusion: the method is portable, but the circuits it finds are not.
Why this matters to the field
Mechanistic interpretability is the branch of AI research that tries to reverse-engineer what specific components inside a neural network are actually doing. The flagship example is the Indirect Object Identification (IOI) circuit, first described in GPT-2 small: a set of name-mover heads, S-Inhibition heads, and induction heads that together let the model complete sentences like "When Mary and John went to the store, John gave the bag to ___" with "Mary."
The classical demonstration identified an attention-head circuit for IOI in GPT-2 small: name-mover heads at late layers, S-Inhibition heads suppressing the subject, prev-token and induction heads in middle layers, and duplicate-token and negative name-mover heads as auxiliary components. That decomposition became canonical -- a template that researchers assumed would generalize.
In mechanistic interpretability, the universality hypothesis asserts that neural networks with different architectures and scales form common internal representations. Strong evidence for or against this hypothesis could significantly affect research priorities: if common representations form across different language models and tasks, then research on small or toy models may be the best way to gain insights into LLM capabilities. This paper is a direct empirical test of that bet.
The experiment: 12 cells, zero matches
Across four composed tasks -- indirect-object identification, greater-than, successor sequences, and variable binding -- and three 1B-class language models from distinct training pipelines (Pythia 1B trained on the Pile, OLMo 1B trained on DCLM, and OLMoE 1B-7B also on DCLM), the authors ran a unified protocol with the matched-random null sampled across ten seeds per cell.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
