Fudan's TinyUSFM Squeezes Ultrasound AI Into 5.5M Parameters Without Losing Accuracy
A knowledge-distilled ultrasound foundation model runs on edge devices with 6% of the parameters while matching its 86M-parameter teacher across 15 organs.
Takeaways
- TinyUSFM distills USFM into a 5.5M-parameter model with 6.4% of the GFLOPs, matching teacher accuracy.
- Achieves 84.91% classification accuracy and 85.78% segmentation Dice across 15 organs on UniUS-Bench.
- Beats vanilla ViT-Tiny by 9.45 points classification and 7.72 points segmentation using the same architecture.
- Feature-gradient coreset selection curates 200K images that outperform training on the full 2.1M dataset.
- Sub-millisecond inference on RTX 2080 Ti, under 1 GB memory, enabling point-of-care ultrasound deployment.
- Code and benchmark released at github.com/MacDunno/TinyUSFM; won first place at MICCAI 2025 IUGC Challenge.
Ultrasound is the workhorse of point-of-care imaging, but the foundation models trained to interpret it are stuck in the datacenter. A team from Fudan University has released TinyUSFM, a 5.5M-parameter student model distilled from the 86.5M-parameter USFM. It runs sub-millisecond inference on a consumer GPU and keeps its teacher's accuracy across 15 organs. The work also introduces UniUS-Bench, the largest publicly available ultrasound benchmark for standardized comparison.
Why shrinking an ultrasound model breaks the usual playbook
Ultrasound images are notoriously operator- and device-dependent. Slight variations in probe angle or position produce significant structural differences even for the same organ, and acquisitions from different vendors introduce large shifts in contrast, resolution, and speckle noise. That is why USFM was pretrained with spatial-frequency dual masking on the 3M-US database of over two million images to become a generalist encoder.
The catch: hundreds of millions of parameters do not fit on a portable ultrasound cart. Existing compression tricks fail here for two reasons. Large-scale ultrasound datasets contain a considerable proportion of low-quality or redundant samples, and directly training lightweight models on such uncurated data leads to underfitting or spurious feature learning. Most existing distillation methods are also designed for natural images and single tasks, so they miss the spectral characteristics that matter for ultrasound.
The two-part recipe
TinyUSFM combines a data-curation stage with an ultrasound-specific distillation objective.

Step 1: Feature-gradient coreset selection. Instead of feeding the student all 2.1M images, the authors pick a 200K subset that is both diverse and informative. They embed every image with the frozen USFM encoder, run a two-level hierarchical K-means (first by organ, then by device variation), and score each sample using gradient statistics collected during the original USFM pretraining. Two metrics are combined: