MambaHSI: Spatial-Spectral Mamba for Hyperspectral Image Classification

TL;DR

MambaHSI introduces a novel spatial-spectral Mamba-based model for hyperspectral image classification, effectively capturing long-range interactions and integrating spatial-spectral information with linear complexity, outperforming existing methods.

Contribution

This work is the first to apply Mamba models at the image level for hyperspectral image classification, combining spatial and spectral modeling in an adaptive framework.

Findings

Outperforms existing HSI classification methods on four datasets.

Demonstrates the effectiveness of spatial-spectral fusion in MambaHSI.

Validates the linear computational complexity of the proposed model.

Abstract

Transformer has been extensively explored for hyperspectral image (HSI) classification. However, transformer poses challenges in terms of speed and memory usage because of its quadratic computational complexity. Recently, the Mamba model has emerged as a promising approach, which has strong long-distance modeling capabilities while maintaining a linear computational complexity. However, representing the HSI is challenging for the Mamba due to the requirement for an integrated spatial and spectral understanding. To remedy these drawbacks, we propose a novel HSI classification model based on a Mamba model, named MambaHSI, which can simultaneously model long-range interaction of the whole image and integrate spatial and spectral information in an adaptive manner. Specifically, we design a spatial Mamba block (SpaMB) to model the long-range interaction of the whole image at the pixel-level. Then, we propose a spectral Mamba block (SpeMB) to split the spectral vector into multiple groups, mine the relations across different spectral groups, and extract spectral features. Finally, we propose a spatial-spectral fusion module (SSFM) to adaptively integrate spatial and spectral features of a HSI. To our best knowledge, this is the first image-level HSI classification model based on the Mamba. We conduct extensive experiments on four diverse HSI datasets. The results demonstrate the effectiveness and superiority of the proposed model for HSI classification. This reveals the great potential of Mamba to be the next-generation backbone for HSI models. Codes are available at https://github.com/li-yapeng/MambaHSI .