Deep Amended Gradient Descent for Efficient Spectral Reconstruction from Single RGB Images

TL;DR

This paper introduces AGD-Net, a novel end-to-end learning framework inspired by gradient descent, for efficient hyperspectral image reconstruction from single RGB images, achieving superior accuracy and efficiency.

Contribution

The paper proposes a physically-interpretable, compact neural network that mimics amended gradient descent, incorporating spectral zero-mean normalization and a rank loss for improved hyperspectral reconstruction.

Findings

Improves reconstruction quality by over 1.0 dB on average.

Reduces parameters by 67 times and FLOPs by 32 times compared to state-of-the-art.

Demonstrates effectiveness across three benchmark datasets.

Abstract

This paper investigates the problem of recovering hyperspectral (HS) images from single RGB images. To tackle such a severely ill-posed problem, we propose a physically-interpretable, compact, efficient, and end-to-end learning-based framework, namely AGD-Net. Precisely, by taking advantage of the imaging process, we first formulate the problem explicitly based on the classic gradient descent algorithm. Then, we design a lightweight neural network with a multi-stage architecture to mimic the formed amended gradient descent process, in which efficient convolution and novel spectral zero-mean normalization are proposed to effectively extract spatial-spectral features for regressing an initialization, a basic gradient, and an incremental gradient. Besides, based on the approximate low-rank property of HS images, we propose a novel rank loss to promote the similarity between the global structures of reconstructed and ground-truth HS images, which is optimized with our singular value weighting strategy during training. Moreover, AGD-Net, a single network after one-time training, is flexible to handle the reconstruction with various spectral response functions. Extensive experiments over three commonly-used benchmark datasets demonstrate that AGD-Net can improve the reconstruction quality by more than 1.0 dB on average while saving 67$\times$ parameters and 32$\times$ FLOPs, compared with state-of-the-art methods. The code will be publicly available at https://github.com/zbzhzhy/GD-Net.