Skip to main content
Back

Ben-Gurion Researchers Push Meta's SAM3 to Map Satellite Images With 100 Labels

Ben-Gurion University researchers benchmarked SAM3 on aerial imagery across four target types, finding hybrid text-plus-box prompts win and 100 annotated objects suffice.

Pro

Paper, Full Access
  • Ben-Gurion University researchers independently benchmarked SAM3 on remote sensing across four target types.
  • Hybrid text-plus-bounding-box prompting won across nearly every metric and target category.
  • Text-only prompting failed completely on dead trees in zero-shot mode, scoring 0.00.
  • Fine-tuning on just 100 objects captured most gains; 200-300 objects offered diminishing returns.
  • SAM3 outperformed a fine-tuned ResUNet baseline on all four remote sensing targets.
  • Persistent Precision-IoU gap reveals boundary delineation as the model's core weakness on overhead imagery.

A team from Ben-Gurion University of the Negev has put Meta's newest concept-driven segmentation model, SAM3, through its paces on a domain it was never trained for: overhead remote sensing imagery. Their benchmark study feeds satellite and aerial photos into a foundation model built on ground-level natural images, and asks a practical question: how much annotation effort do you actually need to make it work?

The short answer is surprisingly little, provided you combine text with bounding boxes. The longer answer says a lot about where vision-language models still stumble when the camera points down instead of forward.

Why overhead imagery breaks foundation models

Segmentation is fundamental to remote sensing, and it is hard for several overlapping reasons: high scene heterogeneity, irregular object boundaries, extreme scale variation, dense spatial patterns, and sensor-specific distortions. On top of that, most foundation models are pretrained on object-centric photos of everyday scenes, so their internal notion of what a "tree" or "building" looks like assumes a side view rather than a bird's-eye one.

SAM3 is Meta's latest segmentation model, and it introduces something new to the SAM family: a concept-driven interface centered on short semantic phrases as primary prompts, unifying textual and geometric inputs within a shared embedding space processed by a ViT-based decoder. In theory you can type "solar panels" and get masks back. The researchers wanted to know whether that promise holds up for aerial photography, and whether a modest amount of fine-tuning can close whatever gap exists.