Pan-Mamba: Effective pan-sharpening with State Space Model

TL;DR

Pan-Mamba introduces a novel pan-sharpening network leveraging the Mamba state space model, enhancing multi-spectral and panchromatic image fusion with superior results across diverse datasets.

Contribution

This work is the first to explore the Mamba model's potential in pan-sharpening, customizing core components for efficient cross-modal information exchange and fusion.

Findings

Outperforms state-of-the-art pan-sharpening methods

Demonstrates superior fusion quality across multiple datasets

Establishes new benchmarks in pan-sharpening performance

Abstract

Pan-sharpening involves integrating information from low-resolution multi-spectral and high-resolution panchromatic images to generate high-resolution multi-spectral counterparts. While recent advancements in the state space model, particularly the efficient long-range dependency modeling achieved by Mamba, have revolutionized computer vision community, its untapped potential in pan-sharpening motivates our exploration. Our contribution, Pan-Mamba, represents a novel pan-sharpening network that leverages the efficiency of the Mamba model in global information modeling. In Pan-Mamba, we customize two core components: channel swapping Mamba and cross-modal Mamba, strategically designed for efficient cross-modal information exchange and fusion. The former initiates a lightweight cross-modal interaction through the exchange of partial panchromatic and multi-spectral channels, while the latter facilities the information representation capability by exploiting inherent cross-modal relationships. Through extensive experiments across diverse datasets, our proposed approach surpasses state-of-the-art methods, showcasing superior fusion results in pan-sharpening. To the best of our knowledge, this work is the first attempt in exploring the potential of the Mamba model and establishes a new frontier in the pan-sharpening techniques. The source code is available at \url{https://github.com/alexhe101/Pan-Mamba}.