SAMST: A Transformer framework based on SAM pseudo label filtering for remote sensing semi-supervised semantic segmentation

TL;DR

SAMST is a semi-supervised remote sensing segmentation framework that combines SAM's zero-shot capabilities with iterative pseudo-label refinement to improve segmentation accuracy with limited labeled data.

Contribution

The paper introduces SAMST, a novel semi-supervised framework that integrates SAM's pseudo-label filtering with self-training for remote sensing segmentation.

Findings

SAMST outperforms baseline methods on Potsdam dataset

Pseudo-label refinement significantly improves segmentation accuracy

The framework effectively utilizes unlabeled remote sensing data

Abstract

Public remote sensing datasets often face limitations in universality due to resolution variability and inconsistent land cover category definitions. To harness the vast pool of unlabeled remote sensing data, we propose SAMST, a semi-supervised semantic segmentation method. SAMST leverages the strengths of the Segment Anything Model (SAM) in zero-shot generalization and boundary detection. SAMST iteratively refines pseudo-labels through two main components: supervised model self-training using both labeled and pseudo-labeled data, and a SAM-based Pseudo-label Refiner. The Pseudo-label Refiner comprises three modules: a Threshold Filter Module for preprocessing, a Prompt Generation Module for extracting connected regions and generating prompts for SAM, and a Label Refinement Module for final label stitching. By integrating the generalization power of large models with the training efficiency of small models, SAMST improves pseudo-label accuracy, thereby enhancing overall model performance. Experiments on the Potsdam dataset validate the effectiveness and feasibility of SAMST, demonstrating its potential to address the challenges posed by limited labeled data in remote sensing semantic segmentation.