Skip to main content
Back

ConvNeXt-V2 With LoRA Beats Larger Medical Imaging Models Using 2.9M Parameters

A tiny LoRA adapter bolted onto a frozen ConvNeXt-V2 backbone outperforms full fine-tuning on endoscopy classification using 3 million trainable parameters.

Pro

  • ConvNeXt-V2 + LoRA beats full fine-tuning on gastrointestinal endoscopy with just 2.9M trainable parameters.
  • Achieves 85.05% accuracy on GastroVision and 98.60% on Kvasir-Capsule, topping DenseNet201 and Inception V3.
  • Freezes the pretrained backbone and only trains two low-rank matrices A and B per layer.
  • Evaluated on 27-class GastroVision and an 11-class subset of Kvasir-Capsule capsule endoscopy.
  • Code released at github.com/sanjay931/peft-gi-recognition, PyTorch, single A100.
  • Attention maps localize lesions more tightly than Inception V3 and DenseNet121 baselines.

Training a separate medical imaging model for every hospital, scanner, or dataset is a well-known tax on research budgets. A new arXiv paper argues that gastroenterologists (and the engineers building tools for them) do not need to pay it. By freezing a pretrained ConvNeXt-V2 vision backbone and inserting low-rank adaptation (LoRA) modules, the authors match or beat much larger baselines on two public endoscopy datasets while training roughly 2.9 million parameters instead of the usual tens of millions.

The setup works on ordinary hardware. Code is released, and the entire training pipeline runs in PyTorch on a single A100.

Why cross-source endoscopy breaks big models

Endoscopic imagery is famously non-uniform. Lighting varies, capsule cameras use different optics, mucosal tissue looks different in the small intestine than in the colon, and classes visually overlap. Video capsule endoscopy focuses on the small intestine, exhibits a distinct mucosal surface characterized by intestinal villi, and produces images with significantly lower resolution and frame rates than conventional endoscopy, with an uncontrolled capsule trajectory. That distribution shift is why teams typically retrain a bespoke model per data source, and it is exactly what the authors want to avoid.

Full fine-tuning a modern backbone for each dataset means storing a full copy of the model weights per task. When the backbone is a ConvNeXt-V2 or a Swin-T hybrid running into the tens of millions of parameters, the storage and retraining bill adds up quickly across hospitals.

Freeze the backbone, train two small matrices