Zhejiang University's MedVeriSeg Teaches Medical AI to Refuse Hallucinated Tumor Masks
A training-free framework catches hallucinated masks in medical segmentation models by reading the geometry of similarity heatmaps between [SEG] tokens and image features.
Takeaways
- MedVeriSeg adds training-free query verification to LISA-like medical segmentation models to reject hallucinated masks. Paper
- Core insight: the [SEG] token similarity heatmap looks compact for real targets and diffuse for absent ones.
- Scores each heatmap on strength, compactness, and purity, then routes ambiguous cases to visual agents.
- Final decision is a weighted score of three evidence sources, not a direct MLLM call, keeping it auditable.
- Hits 83.93% accuracy on MedVeriSeg-Bench (6,022 samples), close to medical-student specialists at 88.23%.
- Code available on GitHub; works as plug-in for MediSee, MedPLIB, and vanilla LISA.
Medical segmentation models built on multimodal LLMs have a quiet reliability problem: ask them to segment a tumor that isn't there, and they will happily draw one anyway. A group from Zhejiang University and Southern Medical University has proposed MedVeriSeg, a training-free verification layer that lets these models say no when the queried target doesn't exist in the image, without retraining a single weight.
The hallucination hiding inside LISA
Most recent MLLM-based medical segmenters follow the LISA blueprint. LISA employs LLaVA to output a text response based on the input image, text query, and a [SEG] token, and the embedding for that customized token is decoded into a segmentation mask via a SAM decoder. In medical variants like MediSee and Med-PLIB, the vision backbone is swapped for MedSAM and the language side for LLaVA-Med, but the mechanism is the same.
These models were trained almost exclusively on valid query-mask pairs, so existing LISA-like methods typically generate masks regardless of whether the target specified in the query is present, leading to hallucinated segmentation. Ask a chest CT model to outline a clavicle in an abdominal slice and you still get a confident mask. In a teaching setting that misleads students; in a clinical setting it risks misdiagnosis.

Retraining with negative samples is expensive and tends to overfit. Feeding the query into a general MLLM like Qwen-3-VL as a gatekeeper also underperforms, because general models struggle with fine-grained medical visual cues.