Multi-view Remote Sensing Image Segmentation With SAM priors

TL;DR

This paper introduces a novel multi-view remote sensing image segmentation method that integrates SAM priors into an Implicit Neural Field to improve performance with limited labels, leveraging scene-wide semantic information.

Contribution

It proposes injecting SAM features into INF and using contrastive pseudo-labeling to enhance multi-view segmentation under limited supervision.

Findings

Outperforms mainstream methods in multi-view segmentation

Effective use of SAM priors improves scene-wide semantic understanding

Demonstrates robustness with limited labeled data

Abstract

Multi-view segmentation in Remote Sensing (RS) seeks to segment images from diverse perspectives within a scene. Recent methods leverage 3D information extracted from an Implicit Neural Field (INF), bolstering result consistency across multiple views while using limited accounts of labels (even within 3-5 labels) to streamline labor. Nonetheless, achieving superior performance within the constraints of limited-view labels remains challenging due to inadequate scene-wide supervision and insufficient semantic features within the INF. To address these. we propose to inject the prior of the visual foundation model-Segment Anything(SAM), to the INF to obtain better results under the limited number of training data. Specifically, we contrast SAM features between testing and training views to derive pseudo labels for each testing view, augmenting scene-wide labeling information. Subsequently, we introduce SAM features via a transformer into the INF of the scene, supplementing the semantic information. The experimental results demonstrate that our method outperforms the mainstream method, confirming the efficacy of SAM as a supplement to the INF for this task.