Skip to main content
Back

Pi-Seg Fixes What CLIP Gets Wrong About Satellite Imagery

A new 170K-image benchmark and a lightweight noise-injection baseline take aim at applying open-vocabulary segmentation to satellite and drone imagery.

  • New OVRSISBenchV2 aggregates 170K images and 128 categories across 10 downstream remote sensing datasets.
  • OVRSIS95K training set provides 95K balanced image-mask pairs across five scene domains.
  • Benchmark adds real GIS tasks: building extraction, road extraction, and flood detection.
  • Pi-Seg baseline injects semantically guided noise into CLIP text and visual features to improve transfer.
  • Reaches 44.40 m-mIoU on V2 with ViT-L, beating GSNet, CAT-Seg, and RSKT-Seg.
  • Code and datasets released on GitHub and Hugging Face.

Open-vocabulary segmentation lets you point a model at an image, type in any category name, and get a pixel mask back. It works well on natural photos thanks to models like CLIP, but on satellite and drone imagery it tends to fall apart. A new paper argues the culprit goes beyond the domain gap to how the field has been measuring progress, and offers both a bigger benchmark and a lightweight baseline to fix it.

The work introduces OVRSISBenchV2, a 170K-image benchmark spanning 128 semantic categories and 10 downstream datasets, along with Pi-Seg, a baseline that improves cross-domain transfer by injecting semantically guided noise into CLIP's feature space. Code and data are on GitHub and Hugging Face.

Why remote sensing keeps breaking OVS models

Natural images are typically captured from an eye-level viewpoint, where objects follow gravity-constrained orientations, so existing OVS models like CLIP do not enforce rotation invariance. Aerial imagery is top-down, cluttered, and full of small objects sharing similar textures. Most existing studies also rely on limited and isolated datasets like subsets of iSAID or Potsdam, with insufficient consideration of class balance and scene diversity. No equivalent of COCO or ADE20K has emerged for the field.