---
title: "Peking University Finds 5% of AI Heads Do All the Visual Thinking"
subtitle: "A new paper identifies a tiny subset of attention heads in multimodal LLMs that do all the visual retrieval work — and shows you can use that to cut inference time in half."
url: "https://alphasignal.ai/news/peking-university-finds-5-of-ai-heads-do-all-the-visual-thinking"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-06-04T00:00:00.000Z"
updated: "2026-06-12T05:08:39.439Z"
type: "Paper"
topics:
  - "Image"
  - "Llms"
subtopics:
  - "Vision Language"
  - "Vision Language"
  - "Hallucinations"
upvotes: 2918
source_url: "https://arxiv.org/abs/2606.05843"
comments: 0
truncated: true
---

# Peking University Finds 5% of AI Heads Do All the Visual Thinking

> A new paper identifies a tiny subset of attention heads in multimodal LLMs that do all the visual retrieval work — and shows you can use that to cut inference time in half.

When a multimodal model answers a question about a video or a document, it has to fish the right visual detail out of a sea of irrelevant pixels. Researchers from Soochow University and Peking University have now [published a paper](https://arxiv.org/abs/2606.05843) revealing exactly how that happens , and the answer is surprisingly concentrated. A tiny, specialized subset of attention heads does almost all the work, while the vast majority of heads just spread attention around without contributing meaningfully to the answer.

## The problem nobody had mapped

Real-world visual inputs are often cluttered with redundant information, and robust multimodal reasoning depends on the model's capacity to selectively retrieve sparse, task-relevant visual cues from complex spatiotemporal scenes. Yet the precise internal mechanisms governing this cross-modal information retrieval remained unexplored, leaving a significant gap in our understanding of multimodal mechanistic interpretability.

Prior work on interpretability of vision-language models had mostly relied on coarse heuristics like spatial entropy or aggregate attention mass, and had been tested only on simple static images. None of it could explain fine-grained, query-conditioned retrieval in dense video or document contexts.

## Introducing CoRe heads and the RAM metric

The researchers introduce a token-level metric called Retrieval Attention Mass (RAM) to identify a highly specialized subset of attention heads they call Context-aware Retrieval (CoRe) heads. Across diverse visual domains and model scales, a clear functional division emerges: CoRe heads act as dedicated information extractors, while most other heads distribute attention over broader contextual regions.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Mechanistic Insights into Functional Sparsity in Multimodal LLMs via CoRe Heads](https://arxiv.org/abs/2606.05843)
- [CoRe Head GitHub Repository](https://github.com/aaxiyao/CoRe_Head)
- [Qwen3-VL Technical Report](https://arxiv.org/abs/2511.21631)