Skip to main content
Back

UC Berkeley Finds Rosetta Neurons Grow Sharper as AI Models Scale

A new scaling law shows that as models grow, a small set of universal neurons becomes sharper and more specialized while the rest stay noisy — with real implications for interpretability and data curation.

  • New scaling law discovered: Rosetta Neurons (universal neurons shared across models) follow a sublinear power law — growing in count but shrinking as a fraction of total neurons as models scale.
  • Neuron Polarization Effect: At scale, universal neurons become sharper and more monosemantic (one concept per neuron), while the rest of the network stays polysemantic and noisy.
  • Covers both language and vision: The effect holds across language models up to 30B parameters and vision models up to 5B, spanning OPT, Pythia, GPT-2, OpenCLIP, and DINOv2.
  • Practical application — data filtering: Rosetta Neuron activations can identify domain-relevant training data, improving continued pretraining efficiency.
  • Methodology: Neurons are matched across models using mutual nearest-neighbor activation correlations; selectivity is measured via excess kurtosis of activation distributions.
  • Code available via the original Rosetta Neurons GitHub repo and the paper on arXiv.

Scaling laws have long told us how loss, compute, and data interact as models grow. But they say almost nothing about what is actually happening inside the network. A new paper from researchers at UC Berkeley and Google DeepMind takes a different approach: instead of measuring outputs, they track individual neurons across model sizes, and what they find is a clean, predictable divergence that no one had formally characterized before.

The paper, "Neuron Populations Exhibit Divergent Selectivity with Scale", is built around a concept called Rosetta Neurons , a class of neurons first identified in the same group's ICCV 2023 work. The core idea: some neurons fire in nearly identical patterns across independently trained models, even when those models have different architectures, training objectives, or data. These common features exist across models with different architectures, different tasks (generative and discriminative), and different types of supervision. Think of them as the universal vocabulary of neural computation , the concepts so fundamental that any sufficiently capable model converges on them.

The Problem With Scaling and Interpretability

Mechanistic interpretability , the project of understanding what individual neurons and circuits actually do , has a well-known enemy: polysemanticity. Neurons often appear to be polysemantic, meaning a single neuron encodes multiple features. This happens because when tasked with representing data with more independent features than it has neurons, a network assigns each feature a linear combination of neurons , a phenomenon called superposition. An LLM is attempting to encode trillions of features such as Python code, DNA, algebraic topology, and cellular biology with far fewer neurons.

The open question was whether scaling up helps or hurts this situation. Do bigger models get messier, cramming even more features into each neuron? Or does something more structured emerge? The answer, it turns out, depends entirely on which neurons you are looking at.