Dual Hyperspectral Mamba for Efficient Spectral Compressive Imaging

TL;DR

This paper introduces Dual Hyperspectral Mamba (DHM), a novel deep learning architecture that effectively captures both global long-range dependencies and local contexts for improved hyperspectral image reconstruction from compressive measurements.

Contribution

The paper proposes DHM with dual hyperspectral S4 blocks, combining global and local modeling to enhance spectral compressive imaging reconstruction performance.

Findings

DHM outperforms existing methods in HSI reconstruction accuracy.

The global hyperspectral S4 block captures long-range dependencies effectively.

Experiments demonstrate the efficiency and robustness of DHM.

Abstract

Deep unfolding methods have made impressive progress in restoring 3D hyperspectral images (HSIs) from 2D measurements through convolution neural networks or Transformers in spectral compressive imaging. However, they cannot efficiently capture long-range dependencies using global receptive fields, which significantly limits their performance in HSI reconstruction. Moreover, these methods may suffer from local context neglect if we directly utilize Mamba to unfold a 2D feature map as a 1D sequence for modeling global long-range dependencies. To address these challenges, we propose a novel Dual Hyperspectral Mamba (DHM) to explore both global long-range dependencies and local contexts for efficient HSI reconstruction. After learning informative parameters to estimate degradation patterns of the CASSI system, we use them to scale the linear projection and offer noise level for the denoiser (i.e., our proposed DHM). Specifically, our DHM consists of multiple dual hyperspectral S4 blocks (DHSBs) to restore original HSIs. Particularly, each DHSB contains a global hyperspectral S4 block (GHSB) to model long-range dependencies across the entire high-resolution HSIs using global receptive fields, and a local hyperspectral S4 block (LHSB) to address local context neglect by establishing structured state-space sequence (S4) models within local windows. Experiments verify the benefits of our DHM for HSI reconstruction. The source codes and models will be available at https://github.com/JiahuaDong/DHM.