Hochschule Darmstadt's DINO ViTs Crack Protein Localization on Tiny Microscopy Datasets
A DINO-pretrained Vision Transformer transfers from Human Protein Atlas images to OpenCell with a macro F1 of 0.860 after fine-tuning.
ProTakeaways
- DINO ViTs pretrained on HPA fluorescence images transfer to OpenCell protein localization with macro F1 up to 0.860 after fine-tuning.
- Zero-shot transfer already hits 0.822 macro F1, showing pretraining alone gives strong representations without task labels.
- Natural channel-wise embedding beats channel replication by roughly 0.03 to 0.07 macro F1 across settings.
- For single-cell kNN evaluation, HPA single-cell pretraining wins with macro F1 ≥ 0.796, beating both HPA FOV and ImageNet-1k.
- Fine-tuning on only 20% of OpenCell slightly underperforms no fine-tuning, hinting at distribution shift from tiny task sets.
- Code and recipe are open at dino4opencell, using weights from Dino4Cells.
Training a deep network from scratch on a few thousand fluorescence microscopy images is a losing battle. Labels are expensive, cell lines are narrow, and every lab uses slightly different stains and channel layouts. A new study out of Hochschule Darmstadt and TH Würzburg-Schweinfurt shows that self-supervised Vision Transformers pretrained on large microscopy or even natural-image datasets can be lifted onto small, task-specific protein-localization problems, provided you handle channel mismatches with care.
Small datasets, big channel headaches
The downstream target here is OpenCell, a CRISPR-tagged imaging project where researchers inserted split-mNeonGreen2 fluorescent tags into 1,310 individual human proteins in HEK293T cells and acquired 6,301 two-channel images (tagged protein and nucleus), with localization manually annotated across 17 subcellular compartments using a three-tier grading system. Tiny by deep learning standards, and the two channels do not match what most pretrained backbones expect.
The pretraining source, the Human Protein Atlas, is much larger. Each field of view was acquired with four channels: microtubule, endoplasmic reticulum, DNA and protein of interest. ImageNet-1k, meanwhile, is three RGB channels of natural photos. So the question the authors chase is whether you can shove a two-channel microscopy image into a ViT that was trained to expect three or four different channels and still get useful features.