Wroclaw Researchers' SinkProbe Beats Every Attention-Based Hallucination Detector in 23 of 28 Tests
A new probe called SinkProbe uses attention sink scores as features to detect hallucinations, beating prior methods on 23 of 28 benchmarks.
ProTakeaways
- SinkProbe detects LLM hallucinations using only attention sink scores fed to a logistic regression classifier.
- Beats prior attention-based detectors on 23 of 28 model-dataset combinations tested.
- Evaluated on Llama3.2-3B, Phi3.5, Llama3.1-8B, and Mistral-Nemo across seven QA and reasoning benchmarks.
- Shows that LLMCheck, LookbackLens, TOHA, and spectral methods all implicitly measure sink behavior.
- Signal is highly localized: only 1 to 4 percent of features carry the hallucination information.
- Main limitation is needing raw attention weights, incompatible with FlashAttention-style serving kernels.
Hallucinations remain the biggest liability of large language models, and a growing body of work has tried to catch them by inspecting what happens inside the network rather than second-guessing the output. A new paper from researchers at Wroclaw University of Science and Technology proposes SinkProbe, a hallucination detector built around a single, mechanistically motivated signal: the attention sink score. It edges out every prior attention-based detector on the majority of benchmarks and, along the way, offers a unifying theory for why several earlier methods worked in the first place.
The sink token, and why it matters here
Attention sinks are tokens that hoover up disproportionate attention from later positions even though they carry little semantic content. The classic example is the <bos> token. First described in the context of streaming inference, sinks are now understood as a load-bearing part of Transformer computation: they prevent over-mixing by providing a stable routing target, and they emerge universally during pretraining.
The authors' central claim is that hallucinations coincide with a breakdown in this routing. When a model hallucinates, attention shifts from distributed, input-grounded computation to compressed, prior-dominated computation. In practice, more attention mass is dumped onto a small number of sink tokens, and the model stops using the actual context.
How SinkProbe works
The pipeline is deliberately lightweight. For every attention head in every layer, the authors compute a per-token sink score, defined as the average amount of attention a token receives from all subsequent tokens, normalized by the remaining sequence length. Then: