Skip to main content
Back

Northwestern Medicine's ConvFormer3D-TAP Hits 96% Accuracy Reading Heart MRI Scans

A hybrid CNN-Transformer trained on 150,974 cine sequences hits 96% accuracy across six cardiac MRI views, cutting downstream biomarker errors.

Pro

Paper, Full Access
  • ConvFormer3D-TAP hits 96% validation accuracy on 150,974 cine CMR sequences across six standard views.
  • Combines 3D convolutional tokenization with a multiscale Transformer bottleneck for local anatomy and long-range cycle dynamics.
  • Phase-aware clip sampling biases training toward high-motion systolic and valve transitions between adjacent views.
  • Masked spatiotemporal reconstruction (up to 60% masking) regularizes the encoder against vendor and protocol shift.
  • Entropy-weighted multi-clip fusion at inference down-weights ambiguous clips, yielding ECE of 0.025 and Brier 0.040.
  • Residual errors cluster in clinically ambiguous pairs (Cine2 vs Cine3, LVOT vs AV); no public code released.

Getting the view right is the unglamorous but foundational step in every cardiac MRI pipeline. Pick the wrong plane and every downstream analysis, from ventricular volumes to strain estimation, inherits the error. A new architecture called ConvFormer3D-TAP tackles this head-on, hitting 96% validation accuracy on a cohort of 150,974 clinically acquired cine sequences spanning six standard cardiac MRI views.

The team behind it, based at Northwestern Medicine's Bluhm Cardiovascular Institute, argues that view classification deserves to be treated as a first-class problem rather than a preprocessing afterthought. Their pitch is a spatiotemporal model that combines 3D convolutions for local anatomy with multiscale self-attention for long-range cardiac cycle dynamics, plus three cine-specific tricks they group under the acronym TAP.

Why picking a plane is harder than it looks

Cine cardiac MRI acquires a stack of orthogonal and oblique planes: two, three, and four chamber long axis views, a short axis stack, and dedicated outflow-tract planes for the left ventricular outflow tract (LVOT) and aortic valve (AV). Manual labeling runs up to 5 minutes per study and shows high inter- and intra-observer variability, and mistakes carry measurable downstream cost.

Misallocation of cine planes has been shown to introduce up to 8% error in left ventricular end-diastolic volume and up to 5% error in ejection fraction. Anatomically adjacent planes look genuinely similar: two-chamber and three-chamber views can differ by a small angulation, and LVOT and AV planes capture nearby slices of the same outflow structure. Prior CNN approaches ignore temporal coherence, and full 3D CNNs tend to overfit on medical datasets while missing long-range temporal dependencies.