M3SR: Multi-Scale Multi-Perceptual Mamba for Efficient Spectral Reconstruction

TL;DR

The paper introduces M3SR, a multi-scale, multi-perceptual architecture based on Mamba and U-Net, that improves spectral reconstruction of hyperspectral images by capturing diverse features more effectively and efficiently.

Contribution

It proposes a novel multi-perceptual fusion block integrated into U-Net for enhanced hyperspectral image reconstruction, addressing limitations of single-scale perception.

Findings

Outperforms existing state-of-the-art methods in spectral reconstruction

Achieves higher accuracy with lower computational cost

Effectively captures global, intermediate, and local features

Abstract

The Mamba architecture has been widely applied to various low-level vision tasks due to its exceptional adaptability and strong performance. Although the Mamba architecture has been adopted for spectral reconstruction, it still faces the following two challenges: (1) Single spatial perception limits the ability to fully understand and analyze hyperspectral images; (2) Single-scale feature extraction struggles to capture the complex structures and fine details present in hyperspectral images. To address these issues, we propose a multi-scale, multi-perceptual Mamba architecture for the spectral reconstruction task, called M3SR. Specifically, we design a multi-perceptual fusion block to enhance the ability of the model to comprehensively understand and analyze the input features. By integrating the multi-perceptual fusion block into a U-Net structure, M3SR can effectively extract and fuse global, intermediate, and local features, thereby enabling accurate reconstruction of hyperspectral images at multiple scales. Extensive quantitative and qualitative experiments demonstrate that the proposed M3SR outperforms existing state-of-the-art methods while incurring a lower computational cost.