---
title: "MBZUAI's Adaptive Conformal Factuality Cuts LLM Hallucination Filter Errors Across Topics"
subtitle: "A new adaptive conformal prediction method learns per-prompt thresholds via quantile regression, fixing over and under filtering of LLM claims across diverse topics."
url: "https://alphasignal.ai/news/mbzuai-s-adaptive-conformal-factuality-cuts-llm-hallucination-filter-errors"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-15T00:00:00.000Z"
updated: "2026-08-23T21:10:52.762Z"
type: "Paper"
topics:
  - "Llms"
  - "Security"
subtopics:
  - "Hallucinations"
  - "Structured Output"
  - "Red Teaming"
upvotes: 3868
source_url: "https://arxiv.org/abs/2604.13991"
comments: 0
truncated: true
---

# MBZUAI's Adaptive Conformal Factuality Cuts LLM Hallucination Filter Errors Across Topics

> A new adaptive conformal prediction method learns per-prompt thresholds via quantile regression, fixing over and under filtering of LLM claims across diverse topics.

Conformal prediction has become one of the more attractive tools for putting statistical guarantees on top of large language models, but it has a stubborn weakness: it treats every prompt the same. A team from MBZUAI is proposing a fix that lets the calibration threshold flex with the input, and the results suggest the standard approach has been quietly miscalibrating hallucination filters on heterogeneous data.

## When one threshold fits nobody

The setup for conformal factuality is straightforward. You take an LLM response, break it into atomic claims, score each claim with some uncertainty measure, and drop the ones that exceed a threshold calibrated on held-out data. In theory, this gives you a statistical guarantee that a certain fraction of the retained claims will be factually correct.

The catch is what kind of guarantee you actually get. [The authors point out](https://arxiv.org/html/2604.13991) that existing conformal procedures for LLMs lack adaptivity: a single calibrated quantile is applied uniformly across all test prompts, regardless of their difficulty, ambiguity, or rarity. That averages out fine across a whole dataset, but for any specific prompt category the threshold can be badly wrong. Easy topics get over-filtered, with too many good claims removed, while hard ones get under-filtered and let hallucinations slip through.

## Rescaling by prompt difficulty

The proposed method, which they call Adaptive Conformal Factuality, keeps the split-conformal machinery intact but adds a learned rescaling step. The pipeline works like this:

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Adaptive Conformal Prediction for Improving Factuality of Generations by Large Language Models](https://arxiv.org/html/2604.13991)
- [LM-Polygraph: Uncertainty Estimation for Language Models](https://aclanthology.org/2023.emnlp-demo.41)
- [LM-Polygraph GitHub repository](https://github.com/IINemo/lm-polygraph)
- [Awesome Conformal Prediction](https://github.com/valeman/awesome-conformal-prediction)