Skip to main content
Back

Princeton Cracks How Transformers Switch Between Memorizing and Generalizing

Princeton researchers reverse-engineer a two-layer transformer and uncover four distinct algorithmic phases plus a new circuit motif they call the task recognition head.

  • Two-layer transformers on Markov chains reveal four algorithmic phases: G1, G2, M1, M2, along memorize/generalize and 1-point/2-point axes.
  • Generalization uses a statistical induction head; memorization uses a newly characterized encoder-pool-decoder task recognition head.
  • Task vectors are causally verified via patching: swapping one chain's task vector into another sequence flips predictions accordingly.
  • Threshold K1* is set by kinetic competition between subcircuits; K2* is set by MLP2's finite representational capacity.
  • Symmetry-constrained SA-transformer predicts induction head formation time scales as N/log(N), verified in the full model.
  • Full code and experiments are open source on GitHub.

In-context learning is one of the most useful and least understood behaviors of transformers. A model trained on enough diverse data can pick up a rule from a handful of examples in its prompt, without any weight updates. A new paper from Princeton's Reddy Lab pulls this behavior apart at the circuit level and shows that transformers use at least two qualitatively different mechanisms to pull it off, with sharp phase transitions between them driven by how diverse the training data is.

The team trained small two-layer transformers on sequences generated by a finite set of discrete Markov chains, then systematically mapped every phase across training time and data diversity K. The setup is simple enough to analyze but rich enough to produce the full menagerie of behaviors people care about in large models: memorization, generalization, abrupt transitions, and induction heads.

Four algorithms hiding in one network

The transformer displays four algorithmic phases, characterized by whether the network memorizes or generalizes, and whether it uses 1-point or 2-point statistics. Think of it as a 2x2 grid. The model either predicts the next token from single-token frequencies (1-point) or from pairwise transitions (2-point), and it either generalizes to unseen Markov chains or memorizes the specific ones it saw during training.

  • G1: generalize using 1-point statistics (stationary distribution)
  • G2: generalize using 2-point statistics (bigram transitions)
  • M1: identify which training chain generated the sequence via 1-point stats, then look up its transition matrix
  • M2: same identification game but using richer 2-point stats

Which phase the network settles into depends on K, the number of distinct Markov chains in the training set. Below a threshold value, the network transitions from G1 to the memorization phases. Above