RS3Mamba: Visual State Space Model for Remote Sensing Images Semantic Segmentation

TL;DR

This paper introduces RS3Mamba, a dual-branch neural network leveraging the novel visual state space model for improved semantic segmentation of remote sensing images, addressing limitations of CNNs and Transformers.

Contribution

It presents the first vision Mamba tailored for remote sensing segmentation, integrating VSS blocks and a collaborative completion module for enhanced global feature modeling.

Findings

Effective on ISPRS Vaihingen dataset

Outperforms existing methods in accuracy

Demonstrates strong generalization on LoveDA dataset

Abstract

Semantic segmentation of remote sensing images is a fundamental task in geoscience research. However, there are some significant shortcomings for the widely used convolutional neural networks (CNNs) and Transformers. The former is limited by its insufficient long-range modeling capabilities, while the latter is hampered by its computational complexity. Recently, a novel visual state space (VSS) model represented by Mamba has emerged, capable of modeling long-range relationships with linear computability. In this work, we propose a novel dual-branch network named remote sensing images semantic segmentation Mamba (RS3Mamba) to incorporate this innovative technology into remote sensing tasks. Specifically, RS3Mamba utilizes VSS blocks to construct an auxiliary branch, providing additional global information to convolution-based main branch. Moreover, considering the distinct characteristics of the two branches, we introduce a collaborative completion module (CCM) to enhance and fuse features from the dual-encoder. Experimental results on two widely used datasets, ISPRS Vaihingen and LoveDA Urban, demonstrate the effectiveness and potential of the proposed RS3Mamba. To the best of our knowledge, this is the first vision Mamba specifically designed for remote sensing images semantic segmentation. The source code will be made available at https://github.com/sstary/SSRS.