---
title: "HKUST's DeP Cuts AI Image Hallucinations by 10 Points Without Retraining"
subtitle: "A training-free method probes multimodal models with paraphrased prompts to catch and correct hallucinations driven by language priors, without touching the image or the weights."
url: "https://alphasignal.ai/news/hkust-s-dep-cuts-ai-image-hallucinations-by-10-points-without-retraining"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-14T00:00:00.000Z"
updated: "2026-08-23T23:41:10.257Z"
type: "Paper"
topics:
  - "Image"
  - "Llms"
subtopics:
  - "Vision Language"
  - "Hallucinations"
  - "Vision Language"
upvotes: 1473
source_url: "https://arxiv.org/abs/2604.12424"
comments: 0
truncated: true
---

# HKUST's DeP Cuts AI Image Hallucinations by 10 Points Without Retraining

> A training-free method probes multimodal models with paraphrased prompts to catch and correct hallucinations driven by language priors, without touching the image or the weights.

Multimodal large language models have a well-known bad habit: they confidently describe things that are not in the image. Ask the same question two slightly different ways and you can flip the answer from correct to nonsense. A new paper from researchers at HKUST (Guangzhou), [Decoding by Perturbation (DeP)](https://arxiv.org/abs/2604.12424), turns that fragility into a diagnostic tool, and uses it to steer the model back toward the pixels.

The core observation is simple. If the meaning of a prompt stays the same, the model's attention to visual evidence should stay stable even when the wording changes. When attention drifts anyway, that shift is a fingerprint of the model leaning on language statistics rather than what it actually sees. DeP treats hallucination as this hypersensitivity and builds a decoding-time fix around it.

![Example showing how small changes to a prompt cause a multimodal model to shift its visual attention and produce different hallucinated answers.](https://arxiv.org/html/2604.12424v1/intro_pic.png) 

## Why prior fixes leave something on the table

Training-free hallucination mitigation has mostly gone down two roads, and both carry real costs. Visual-degradation methods add noise or corruption to the image and contrast the outputs, which pulls the model away from the natural image distribution it was trained on and makes it hard to localize what actually triggered the false answer. Attention-intervention methods reach into the decoder to rebalance vision and language, but the intrusive edits tend to hurt fluent generation.

DeP sidesteps both routes. The image stays untouched, the weights stay untouched, and the intervention happens on the text side and in the output space where the prior actually lives.

## How the probe works

At each decoding step, DeP generates a small set of paraphrased prompts that preserve meaning but vary the surface phrasing. Given the original prompt, it constructs a perturbation set of M variations and runs them through the model to collect logits and cross-modal attention. The perturbations come from three progressively aggressive strategies:

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Decoding by Perturbation: Mitigating MLLM Hallucinations via Dynamic Textual Perturbation](https://arxiv.org/abs/2604.12424)
- [DeP full paper (HTML)](https://arxiv.org/html/2604.12424)
- [POPE: Evaluating Object Hallucination in Large Vision-Language Models](https://arxiv.org/abs/2305.10355)