TAC Turns AI Uncertainty Scores Into Real Hallucination Detectors
A new post-hoc calibration method called Truth AnChoring maps raw LLM uncertainty scores to probabilities grounded in actual correctness.
- Paper formalizes proxy failure: why LLM uncertainty metrics behave unstably across datasets and models.
- Proves AUC of any score is bounded by its mutual information with correctness.
- Truth AnChoring (TAC) maps raw uncertainty scores to truth-aligned probabilities with a small MLP.
- Cuts ECE by 17 to 27 points on TriviaQA, SciQ, PopQA across five open LLMs.
- Works with as few as 32 labels or 30% corrupted supervision, outperforming CUE without extra features.
- Code and pipeline available at github.com/ponhvoan/TruthAnchor under Apache-2.0.
A paper accepted to Findings of EMNLP formalizes why common uncertainty metrics can fail at hallucination detection. Researchers from Nanyang Technological University, KAIST, and Shanghai Jiao Tong University call the mismatch proxy failure and propose Truth AnChoring, or TAC, as a lightweight calibration method.
Hallucination detectors often treat predictive entropy, perplexity, semantic entropy, and similar scores as evidence that an answer is correct. These metrics describe model behavior rather than factual accuracy. TAC uses labeled examples to convert a raw score into an estimated probability of correctness, giving developers a value that can support abstention, retrieval, review, or routing thresholds.
- Input: An uncertainty score produced for a model response.
- Output: An estimated probability that the response is correct.
- Training data: Binary correctness labels paired with uncertainty scores.
- Reported scope: Fact-based question answering with models from 3B to 9B parameters.
Entropy can miss confident errors
Predictive entropy measures how dispersed the model’s next-token distribution is. A low value indicates that the model strongly prefers a continuation, but that continuation can still express a false claim. Semantic entropy broadens the analysis by grouping responses according to meaning, yet it remains a behavioral proxy whose relationship with correctness can shift across models, prompts, and datasets.
The authors formalize proxy failure with two propositions:
- Information limits discrimination. A score’s AUC is bounded by the mutual information between that score and correctness. When the score contains little information about truth, it cannot reliably rank correct answers above incorrect ones.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
