Skip to main content
Back

LLMs' Confidence Scores Actually Reveal Their Internal Answer Probabilities

A new study finds that when an LLM states a confidence in words, that number is tightly linked to the probability distribution over its next token.

Paper, Full Access
  • New paper shows verbalized LLM confidences are statistically coupled to internal token probabilities, not just surface text.
  • Authors are Williamson, Li, Foti, Webb, and Fedzechkina, posted on arXiv.
  • Both readouts shift when training-data frequencies change, confirming shared underlying representation.
  • Explicit confidence phrases in training text can bias both readouts, causing miscalibration.
  • Experiments span LoRA, full fine-tuning, in-context learning, and from-scratch training on synthetic data.
  • Implication: verbalized confidence is a usable probe for closed, API-only models without logit access.

When a language model reports 70 percent confidence, that number may be statistically connected to the probabilities shaping its output. In an arXiv preprint, Sinead Williamson, Jiaxuan Li, Nick Foti, Russ Webb, and Masha Fedzechkina find that verbalized confidence moves with a model’s internal probability distribution, including after they control for shared influences from the training data.

For developers working with closed models, verbalized confidence offers an accessible uncertainty signal. It requires no logits, embeddings, or model weights. The findings suggest that a prompted confidence score can provide a noisy proxy for internal probabilities, provided teams validate and calibrate it for their specific model and task.

Two routes to a probability

Language models expose uncertainty through two distinct readouts:

Readout How it is obtained What it captures
Internal probability Derived from logits, the raw scores converted into probabilities for candidate tokens or answers The model’s relative preference among possible continuations
Verbalized probability Generated after prompting the model to state a confidence word or number A textual claim about the model’s certainty

Earlier research indicates that internal probabilities respond to relative frequencies in training data, while verbalized probabilities respond to explicit claims such as “there is a 70 percent chance.” Because frequency and asserted probability often coincide, a correlation between the two readouts could arise from common training signals. The authors separate those signals to test whether an additional connection remains.