Unleashing Correlation and Continuity for Hyperspectral Reconstruction from RGB Images

TL;DR

This paper introduces a novel neural network architecture that leverages local spectral correlation and global spectral continuity to improve hyperspectral image reconstruction from RGB images, achieving state-of-the-art results.

Contribution

The paper proposes CCNet, a new model combining spectral correlation and continuity modules with adaptive fusion, advancing hyperspectral reconstruction from RGB images.

Findings

Achieves state-of-the-art performance on NTIRE2022 and NTIRE2020 datasets.

Effectively models local spectral similarity and global spectral variation.

Outperforms existing algorithms in spectral reconstruction quality.

Abstract

Reconstructing Hyperspectral Images (HSI) from RGB images can yield high spatial resolution HSI at a lower cost, demonstrating significant application potential. This paper reveals that local correlation and global continuity of the spectral characteristics are crucial for HSI reconstruction tasks. Therefore, we fully explore these inter-spectral relationships and propose a Correlation and Continuity Network (CCNet) for HSI reconstruction from RGB images. For the correlation of local spectrum, we introduce the Group-wise Spectral Correlation Modeling (GrSCM) module, which efficiently establishes spectral band similarity within a localized range. For the continuity of global spectrum, we design the Neighborhood-wise Spectral Continuity Modeling (NeSCM) module, which employs memory units to recursively model the progressive variation characteristics at the global level. In order to explore the inherent complementarity of these two modules, we design the Patch-wise Adaptive Fusion (PAF) module to efficiently integrate global continuity features into the spectral features in a patch-wise adaptive manner. These innovations enhance the quality of reconstructed HSI. We perform comprehensive comparison and ablation experiments on the mainstream datasets NTIRE2022 and NTIRE2020 for the spectral reconstruction task. Compared to the current advanced spectral reconstruction algorithms, our designed algorithm achieves State-Of-The-Art (SOTA) performance.