Hyperspectral Demosaicing of Snapshot Camera Images Using Deep Learning

TL;DR

This paper introduces a deep learning-based demosaicing method for hyperspectral snapshot camera images, utilizing a new dataset and outperforming existing neural network approaches in reconstructing high-quality spectral images.

Contribution

It presents a novel parallel neural network architecture trained on a new hyperspectral dataset with real ground truth, improving demosaicing accuracy for snapshot hyperspectral cameras.

Findings

The proposed network outperforms state-of-the-art demosaicing networks.

A new ground truth dataset for hyperspectral demosaicing is created.

The method effectively reduces chromatic artifacts in reconstructed images.

Abstract

Spectral imaging technologies have rapidly evolved during the past decades. The recent development of single-camera-one-shot techniques for hyperspectral imaging allows multiple spectral bands to be captured simultaneously (3x3, 4x4 or 5x5 mosaic), opening up a wide range of applications. Examples include intraoperative imaging, agricultural field inspection and food quality assessment. To capture images across a wide spectrum range, i.e. to achieve high spectral resolution, the sensor design sacrifices spatial resolution. With increasing mosaic size, this effect becomes increasingly detrimental. Furthermore, demosaicing is challenging. Without incorporating edge, shape, and object information during interpolation, chromatic artifacts are likely to appear in the obtained images. Recent approaches use neural networks for demosaicing, enabling direct information extraction from image data. However, obtaining training data for these approaches poses a challenge as well. This work proposes a parallel neural network based demosaicing procedure trained on a new ground truth dataset captured in a controlled environment by a hyperspectral snapshot camera with a 4x4 mosaic pattern. The dataset is a combination of real captured scenes with images from publicly available data adapted to the 4x4 mosaic pattern. To obtain real world ground-truth data, we performed multiple camera captures with 1-pixel shifts in order to compose the entire data cube. Experiments show that the proposed network outperforms state-of-art networks.