Skip to main content
Back

Researchers Expose How Radiology AI Fails on Blurry Real-World X-Rays

Researchers build a benchmark for noisy chest X-rays and a training method that keeps report generation models honest when image quality drops.

Pro

Paper, Full Access
  • New paper introduces the LRRG benchmark for radiology report generation on low-quality chest X-rays.
  • Automated Quality Assessment Agent uses Qwen-VL-30B plus DICOM Deviation Index, NIQE, MUSIQ, CLIPIQA and TOPIQ.
  • Splits MIMIC-CXR into Standard, Mildly Suboptimal, and Severely Suboptimal to isolate quality as a variable.
  • Dual-loop training uses bi-level optimization to align gradients across quality regimes.
  • Evaluated against EKAGen, Mamba_Xray, GPT baseline, and Qwen3-VL 4B and 8B with and without fine-tuning.
  • Code and data promised on acceptance; task restricted to single frontal view with no prior history.

Most vision language models that write radiology reports have quietly been graded on easy mode. They train and evaluate on clean, well-exposed chest X-rays, then get dropped into hospitals where a nontrivial share of scans are underexposed, motion-blurred, or poorly positioned. A new arXiv paper tackles that gap directly, showing that current models fall apart on suboptimal images and proposing both a benchmark to measure the damage and a training recipe to close it.

The authors frame the problem cleanly. Vision-language models have advanced automated radiology report generation, but existing methods implicitly assume high-quality inputs and overlook the noise and artifacts prevalent in real clinical environments, so current systems degrade sharply on suboptimal scans. Models score well on curated test sets and much worse on the messy images that actually show up in a radiologist's queue.

Comparison of severely suboptimal, standard, and retake chest X-rays with clinical findings

Chest X-ray quality is a moving target

Chest radiography quality is not a single number. It depends on exposure dose, patient positioning, breathing artifacts, and whether the anatomy that matters (costophrenic angles, apices, mediastinum) is actually in the frame. Radiologists routinely order retakes when a scan is unreadable, but those retake pairs are rare in public datasets, which makes it hard to study quality-driven failures at scale.

To work around that, the paper introduces an Automated Quality Assessment Agent (AQAA) that scores every image in the MIMIC-CXR corpus and sorts it into Standard, Mildly Suboptimal, or Severely Suboptimal. The agent is a stack of signals fed to a multimodal LLM, Qwen-VL-30B, bridging low-level pixel statistics and high-level clinical interpretation by integrating expert priors.