DTU's Hybrid Vision Mamba Model Hits 94.2% Accuracy Detecting Breast Cancer
A CNN plus State Space Model hybrid classifies breast lesions on CBIS-DDSM with 0.875 AUC, beating Swin, ViT, and EfficientNet baselines.
ProTakeaways
- Hybrid EfficientNetV2-M plus Vision Mamba classifies mammography ROIs on CBIS-DDSM.
- Reaches AUC 0.875, accuracy 94.2%, sensitivity 0.89, specificity 0.95 on the test split.
- Beats re-implemented ResNet-50, ViT-B/16, Swin-T, EfficientNet-B3, and CMT-Small baselines under identical preprocessing.
- Ablation shows the hybrid gains 2.5% AUC over EfficientNetV2-M alone at 32.4M parameters total.
- Two-stage training freezes CNN backbone first, then fine-tunes end to end with AdamW.
- Limitations: ROI-based single-view crops only, not full-image screening; no code or weights released.
A team from Delhi Technological University has put together a hybrid vision model that pairs a convolutional backbone with Vision Mamba, a State Space Model, to classify breast lesions in mammograms as benign or malignant. On cropped regions of interest from the CBIS-DDSM dataset, the model reaches an AUC of 0.875 and 94.2% accuracy, edging out ResNet, ViT, Swin, and EfficientNet baselines re-implemented under the same preprocessing pipeline. The paper reads as a case study in why linear-complexity sequence models are becoming attractive for high-resolution medical imaging.
Where CNNs and Transformers both stumble
Mammography interpretation is hard for well-known reasons: high breast density can obscure lesions, and subtle morphological signs of early-stage cancers get missed, producing inter-observer variability among radiologists. CNNs like EfficientNet automated feature extraction and pushed accuracy close to human experts, but their localized receptive fields limit any ability to model long-range spatial dependencies. That matters for abnormalities like architectural distortions, where a lesion in one region warps tissue patterns elsewhere on the image.
Vision Transformers address the global-context problem with self-attention, but the cost of computing attention grows with the square of the number of image patches, which becomes painful at mammography resolutions. That opens a lane for State Space Models like Mamba and its vision-centric variant, Vision Mamba (ViM), which model long sequences with linear complexity (O(N)) and offer Transformer-like global reach without the compute bill. In plain terms, Mamba scans a sequence and maintains a compact internal state that summarizes everything seen so far, sidestepping the all-pairs comparison that makes attention expensive.