Skip to main content
Back

HG-DTGL Trains Medical AI by Tracking Where Radiologists Look

A new semi-supervised framework treats radiologist eye movements as a second teacher, cutting labeling needs while beating state-of-the-art on four segmentation benchmarks.

Pro

Paper, Full Access
  • HG-DTGL adds radiologist eye-tracking as a hidden second teacher inside the mean-teacher semi-supervised framework.
  • GazeMix crops gaze-centered patches and pastes them onto unlabeled images for reliable data augmentation.
  • Reaches 89.37% Dice on ACDC and 86.41% on CAMUS using only 5% labeled data.
  • GazeMix alone lifted Dice by up to 15.87% in ablation experiments over the baseline.
  • Validated across ten organs on ACDC, CAMUS, Synapse and SCR covering MRI, ultrasound, CT and X-ray.
  • Published in Neural Networks; no public code release announced.

Labeling medical images pixel by pixel is one of the most expensive bottlenecks in clinical AI. A radiologist can spend hours outlining a single organ, and that cost multiplies fast when you need thousands of scans. A new paper proposes a practical shortcut: instead of drawing masks, let the radiologist look at the image and record where their eyes go.

The method, called Human Gaze-based Dual Teacher Guidance Learning (HG-DTGL), plugs eye-tracking data into the classic mean-teacher semi-supervised framework as a second, hidden teacher. Compared with manual annotation, gaze data is easier and cheaper to obtain. Trained with as little as 5% labeled data, the model beats prior semi-supervised baselines across ten different organs and tissues spanning MRI, ultrasound, CT, and X-ray.

Why gaze is a useful signal

Semi-supervised segmentation typically leans on the mean-teacher setup: a student network learns from both a small labeled set and pseudo-labels produced by a teacher network that is an exponential moving average of the student. It works, but the teacher only knows what the student already sort of knows. When medical targets have low contrast against the background, both models can drift toward the wrong region.

Comparison of human gaze patterns versus neural network attention maps on cardiac MRI scans

Gaze fixes this by injecting a strong human prior about where to look. The gaze teacher guides the model toward regions a clinician would fixate on, while the network teacher drives learning on the semantic content inside those regions via its pseudo-labels. Together they give the student two complementary signals: spatial attention from the human and semantic content from the network.