Skip to main content
Back

Why LLMs Lie About Confidence and How to Actually Fix It

A systematic review and benchmark of six uncertainty quantification methods across seven modern LLMs reveals which calibration tricks actually work.

Pro

  • Systematic review benchmarks six uncertainty quantification and calibration methods on seven modern LLMs.
  • Code and review list are open on GitHub.
  • Combining chain-of-thought with verbalized uncertainty gave the most consistent calibration gains across models.
  • Temperature and Platt scaling help Qwen2.5 and Mistral out-of-domain but not Llama 3.1.
  • Natural Questions caused a 15 to 22% AUROC drop, exposing brittleness under distribution shift.
  • Heavy RLHF can flatten confidence into overconfidence, a phenomenon the authors call the alignment tax.

Hallucination remains the most stubborn failure mode of modern language models, and the usual defense (asking the model how confident it is) tends to be wildly unreliable. A new systematic review takes on the messy question of how to actually measure and fix that miscalibration, pairing a survey of the field with a hands-on benchmark of six representative methods across seven LLMs.

The paper pulls together work on uncertainty quantification (UQ) and calibration that has been scattered across NLP, Bayesian deep learning, and conformal prediction, then puts the leading techniques head to head on generative question answering. The authors also released their code and review list.

Why confidence scores lie

Calibration means that when a model says it is 80% sure, it should be right about 80% of the time. LLMs are notoriously bad at this, drifting into over-confidence or under-confidence depending on the prompt. The paper splits the underlying uncertainty into two flavors that the community keeps conflating:

  • Epistemic (model) uncertainty: gaps in what the model has learned. Reducible with more data or a better architecture.
  • Aleatoric (data) uncertainty: ambiguity or noise in the input itself. Considered the root of hallucination and largely irreducible.

In practice, the authors note that LLM generation muddies the water. Prompt underspecification, decoding stochasticity, and semantic ambiguity all bleed into both categories at once, so any clean epistemic/aleatoric split is more of an analytical lens than a hard boundary.