Skip to main content
Back

Stanford and Harvard Finally Explain Why Bigger AI Models Always Win

A new paper from Stanford, Harvard, MIT, and Anthropic finally explains why bigger AI models outperform smaller ones — and it comes down to gradient interference, not just capacity.

  • Core finding: Larger models learn rare/complex tasks not because of greater expressivity, but because they reduce gradient interference between common and rare tasks during training.
  • Mechanism: Small models' neurons are monopolized by high-frequency tasks; their gradient updates continuously overwrite any rare-task learning before it can accumulate.
  • Proven analytically: A formal theorem shows models learn features in strict order of "utility" (frequency × complexity weight) — rare tasks only get neurons once common tasks are fully satisfied.
  • Validated on real LLMs: OLMo models from 4M to 4B parameters were pretrained on Dolma v1.7 with injected novel tasks; only larger models learned low-frequency tasks and showed less gradient interference.
  • Practical implication: Increasing a target task's frequency in the training data mix may be more compute-efficient than scaling model size for learning rare capabilities.
  • Memorization reframed: Retaining partial task memories across sparse observations is a prerequisite for generalization on rare tasks, connecting to grokking dynamics.

The AI field has operated on a simple empirical rule for years: bigger models are better. But why has remained surprisingly murky. A new paper from researchers at Stanford, Harvard, MIT, Goodfire, and Anthropic offers the most mechanistic explanation yet, and the answer is not what most people assumed. It is not simply that larger models can express more functions. The bottleneck is in the training dynamics , specifically, a data-induced competition for neurons that systematically starves rare and complex tasks of the signal they need to be learned.

The question everyone ignored

Scaling laws have been the backbone of modern AI development. These "scaling laws" describe predictable relationships between model size, computational resources, and performance , when developers increase training compute by 10x, model performance tends to improve by a predictable amount across diverse tasks. But these laws describe what happens, not why. The standard intuition is that larger models have more parameters, so they can represent more complex functions. The new paper, "Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention", argues this expressivity framing misses the real story.

The authors develop a phenomenological argument that power-law scaling already suggests that a larger model will be able to learn a part of the data distribution that a smaller model fails to learn, even with infinite training data. That last clause is the key insight: this is not a sample efficiency argument. Even if you train a small model forever on infinite data, it will still fail to learn certain tasks that a larger model picks up. The bottleneck is structural, not statistical.

Loss curves showing compute-optimal vs asymptotic scaling, illustrating the regime where only model scaling can close the gap

Neurons as a scarce resource

The core mechanism the paper identifies is data-induced resource competition. Think of a model's neurons as a shared budget. During training, every gradient update is a vote for which tasks get to occupy that budget. High-frequency and low-complexity tasks generate strong, frequent gradient signals, so they win the vote , repeatedly. Rare and complex tasks generate weak, infrequent signals, and they keep getting outbid.

Smaller models allocate their neurons to high frequency or low complexity tasks, and so they learn solutions that perform poorly on rare and complex tasks , and this happens even when solutions capable of expressing the desired task exist. This is the crucial point: the small model is not incapable of representing the rare task in principle. It just never gets trained to do so, because the gradient updates from common tasks continuously overwrite whatever progress was made on rare ones.