Deep-learning-based Hyperspectral imaging through a RGB camera

TL;DR

This paper investigates how the spectral sensitivity of RGB cameras affects hyperspectral image reconstruction and introduces a novel neural network architecture that enhances reconstruction accuracy using 3D matrix transpose.

Contribution

It highlights the importance of spectral sensitivity calibration and proposes a new CNN-based method with a modified data structure for better HSI reconstruction.

Findings

CSS significantly impacts reconstruction accuracy

Calibration improves HSI quality

Proposed network outperforms existing methods

Abstract

Hyperspectral image (HSI) contains both spatial pattern and spectral information which has been widely used in food safety, remote sensing, and medical detection. However, the acquisition of hyperspectral images is usually costly due to the complicated apparatus for the acquisition of optical spectrum. Recently, it has been reported that HSI can be reconstructed from single RGB image using convolution neural network (CNN) algorithms. Compared with the traditional hyperspectral cameras, the method based on CNN algorithms is simple, portable and low cost. In this study, we focused on the influence of the RGB camera spectral sensitivity (CSS) on the HSI. A Xenon lamp incorporated with a monochromator were used as the standard light source to calibrate the CSS. And the experimental results show that the CSS plays a significant role in the reconstruction accuracy of an HSI. In addition, we proposed a new HSI reconstruction network where the dimensional structure of the original hyperspectral datacube was modified by 3D matrix transpose to improve the reconstruction accuracy.