ETH Zurich's MVMAE Beats Larger X-ray Models Using 10x Fewer Labels
A self-supervised method treats multiple X-ray projections of the same patient as paired data, beating supervised and vision-language baselines on three datasets.
- MVMAE combines masked image reconstruction with cross-view alignment across projections within a radiology study.
- Beats supervised, BiomedCLIP, and CheXagent baselines on MIMIC-CXR, CheXpert, and PadChest.
- Hits 80% AUROC with 5k labels; supervised baseline needs 50k for the same.
- MVMAE-V2T adds report captioning during pretraining, boosting results in low-label regimes.
- Text is used only during pretraining; inference remains pure vision encoder.
- Code at github.com/agostini335/MVMAE, accepted to TMLR 2026.
Researchers at ETH Zurich, UC Irvine, and the University of Zurich have released MVMAE, a pretraining method that learns from the natural grouping of images within radiology studies. In the paper, Structure Is Supervision: Multiview Masked Autoencoders for Radiology, the authors report higher disease-classification AUROC than supervised and vision-language baselines across MIMIC-CXR, CheXpert, and PadChest.
Most chest X-ray pipelines treat each image as an independent sample. MVMAE preserves study membership, allowing frontal views, lateral views, and repeated acquisitions from the same examination to supervise one another. The method requires no additional clinical labels and produces a vision encoder that can process a single image at inference.
Hidden labels in the study structure
MIMIC-CXR, CheXpert, and PadChest contain hundreds of thousands of studies, far fewer samples than the billions used to train general-purpose vision models. Their diagnostic labels are expensive, can contain noise, and vary with institutional reporting practices. These constraints limit conventional supervised training, particularly when only a small labeled subset is available.
A study’s images already share anatomy, pathology, and acquisition context. Radiologists use the relationship between projections to localize findings and resolve overlapping structures. In machine-learning terms, two views from the same examination form a positive pair: their pixels differ, but their representations should preserve consistent clinical information.
Two losses, one study-level encoder
A conventional masked autoencoder hides most image patches, encodes the visible remainder, and reconstructs the missing pixels. MVMAE retains that objective and adds a loss that aligns representations from images belonging to the same study.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
