Skip to main content
Back

Rutgers Cracks Open Vision Transformers Using Cross-Layer Transcoders

A new interpretability framework decomposes Vision Transformer representations layer by layer, exposing where CLIP actually stores its visual knowledge.

Paper, Full Access
  • Cross-Layer Transcoders replace ViT MLP blocks with sparse, interpretable proxies without hurting CLIP zero-shot accuracy.
  • Trained on CLIP ViT-B/32 and ViT-B/16 across CIFAR-100, COCO, and ImageNet-100 with three sparsifier variants.
  • Reconstruction reaches cosine similarity 0.92 to 0.97 and R-squared up to 0.95 across all layers.
  • Patch tokens show diagonal-dominant attribution; the [CLS] token aggregates credit broadly across depth.
  • Keeping only the top-4 out of 12 source layers recovers baseline accuracy on ImageNet-100.
  • No public code or checkpoints released yet; attention heads are not modeled.

Vision Transformers power everything from CLIP to Segment Anything, but nobody can really explain what happens inside them. A group of researchers from Rutgers and UC San Diego just proposed a way to crack them open by borrowing a technique that has been gaining traction in language model interpretability: Cross-Layer Transcoders (CLTs). The result is a sparse, depth-aware proxy that can literally replace the MLP blocks inside a ViT without breaking classification, while also revealing which layers do the semantic heavy lifting.

Why single-layer probes fall short

The dominant tool for peeking inside transformers has been the Sparse Autoencoder (SAE), which learns an overcomplete dictionary of interpretable features from a layer's activations. The catch: SAEs operate one layer at a time and miss the cross-layer computational structure of Transformers, along with the relative significance of each layer in shaping the final representation. You end up with a snapshot of what a layer knows, but no story of how the network builds that knowledge across depth.

CLTs were introduced in the language domain by Anthropic as part of their circuit tracing work, but nobody had seriously tested them on vision models, which behave very differently. ViTs juggle a global [CLS] token alongside spatially structured patch tokens, and it was not obvious the same trick would translate.