TU Munich's FINER Catches Vision AI Hallucinating Subtle Details With 24% Fix
New benchmarks show frontier vision language models confidently confirm subtle false claims buried in long queries, and a DPO recipe cuts hallucinations by up to 24.2%.
ProTakeaways
- FINER benchmarks reveal MLLMs hallucinate when subtle wrong details are buried in otherwise correct long queries
- InternVL3.5-14B accuracy drops from ~80% to ~20% as query granularity increases on FINER-CompreCap
- Four question types tested: multi-object, multi-attribute, multi-relation, and "what" questions
- FINER-Tuning uses DPO on minimally edited negative queries, generated entirely with open-source Phi-4-14B
- Up to 24.2% gain on FINER, plus improvements on 8 hallucination and 6 general benchmarks with no alignment tax
- Code, data, and tuned model checkpoints released; CVPR 2026 Oral
Ask a frontier multimodal model whether an image contains a cat, and it will nail the answer. Ask it whether the image contains a fluffy orange cat sitting on a wooden chair next to a blue mug with a chipped handle, and things fall apart. A new CVPR 2026 Oral paper from TU Munich, Helmholtz Munich, Google, and Télécom Paris shows that as visual questions get more detailed, current MLLMs increasingly rubber-stamp claims that are subtly wrong, even when the error is buried inside otherwise accurate description.
The authors call these Fine-grained Negative Queries (FINER), release two benchmarks that stress-test this failure mode, and propose FINER-Tuning, a Direct Preference Optimization recipe that reduces hallucinations without the usual alignment tax.
The failure mode nobody was measuring
As a query becomes more detailed, the base model grows much more likely to answer "Yes" to claims that should be rejected. For InternVL3.5-14B, accuracy drops from around 80% to around 20% on FINER-CompreCap, and to around 15% on FINER-DOCCI. The pattern holds across four frontier MLLMs the authors evaluated.

MLLMs hallucinate when fine-grained mismatches co-occur with genuinely present elements in the image. If nine details in a query are correct and one is subtly wrong, the model latches onto the accurate context and misses the anomaly. Existing hallucination benchmarks such as POPE, CHAIR, and AMBER largely test coarse yes/no object questions, so this failure mode has been flying under the radar.