Stanford's MAttr Tops AI Interpretability Benchmark by Nearly 3x
A new interpretability method learns which internal components matter for a behavior, and pinpoints just 1% of Llama 3.1 weights driving refusals.
- Stanford team introduces Matryoshka Attribution, framing interpretability as learning a ranking via gradient descent.
- Uses a differentiable sigmoid top-k mask with randomized sparsity, no straight-through or Gumbel tricks needed.
- Ranked #1 on the Mechanistic Interpretability Benchmark, roughly 2.9x the runner-up score.
- Reverting just 1% of Llama 3.1 8B Instruct weights to base disables refusals while keeping capabilities.
- Rankings transfer across tasks: 86% performance recovery from subtraction to modular addition.
- Code released for both circuit-level and parameter-level attribution.
Matryoshka Attribution learns one ranking for every circuit size
A Stanford-led team has introduced Matryoshka Attribution, or MAttr, a method for ranking the internal components responsible for a model behavior. It achieved the highest score on the node track of the Mechanistic Interpretability Benchmark. A single training run produces a ranking that can be sliced at many sparsity levels, reducing repeated optimization and making circuits easier to compare across sizes.
The paper, by Aryaman Arora and collaborators including Noah Goodman, Dan Jurafsky, and Christopher Potts, formulates attribution as an optimization problem over nested component sets. The method applies to attention heads, MLP blocks, neurons, sparse autoencoder features, and parameter updates between checkpoints.
Why localization gets expensive
A language model distributes computation across many interacting components. Attention heads move information between token positions, while MLP blocks transform each position’s representation. Researchers localize a behavior by measuring which components preserve, weaken, or remove it under intervention.
| Approach | Mechanism | Main trade-off |
|---|---|---|
| Causal ablation | Disable components and measure the behavioral change | Provides direct intervention evidence, but combinations become expensive to test |
| Gradient attribution | Differentiate a behavior score with respect to components | Runs quickly, but measures local sensitivity rather than the full effect of removal |
| Fixed-budget mask learning | Optimize a selector for a chosen circuit size | Can find compact circuits, but often requires separate runs for different sizes |
| MAttr | Optimize one shared ranking while sampling circuit sizes | Covers many sparsity levels in one run, but still requires task-specific training |
One ranking, many circuit sizes
MAttr assigns a learnable score to every candidate component. A differentiable sigmoid top-k operator converts those scores into a mask for a selected budget, allowing gradients to update the ranking during training. Evaluation sorts the scores and selects the top
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
