NTIRE 2020 Challenge on Spectral Reconstruction from an RGB Image

TL;DR

This paper reviews the NTIRE 2020 challenge on reconstructing hyperspectral images from RGB images, introduces a new large dataset, and evaluates state-of-the-art methods in spectral reconstruction.

Contribution

It presents a comprehensive challenge framework, a new large hyperspectral dataset, and an extensive evaluation of top methods for spectral reconstruction from RGB images.

Findings

Top methods significantly improved spectral reconstruction accuracy.

The new dataset enables better training and benchmarking.

Challenges remain in real-world noisy and uncalibrated scenarios.

Abstract

This paper reviews the second challenge on spectral reconstruction from RGB images, i.e., the recovery of whole-scene hyperspectral (HS) information from a 3-channel RGB image. As in the previous challenge, two tracks were provided: (i) a "Clean" track where HS images are estimated from noise-free RGBs, the RGB images are themselves calculated numerically using the ground-truth HS images and supplied spectral sensitivity functions (ii) a "Real World" track, simulating capture by an uncalibrated and unknown camera, where the HS images are recovered from noisy JPEG-compressed RGB images. A new, larger-than-ever, natural hyperspectral image data set is presented, containing a total of 510 HS images. The Clean and Real World tracks had 103 and 78 registered participants respectively, with 14 teams competing in the final testing phase. A description of the proposed methods, alongside their challenge scores and an extensive evaluation of top performing methods is also provided. They gauge the state-of-the-art in spectral reconstruction from an RGB image.