Hyperspectral recovery from RGB images using Gaussian Processes

TL;DR

This paper introduces a Bayesian Gaussian Process-based method to recover detailed hyperspectral information from RGB images by modeling spectral smoothness and non-negativity, validated on multiple datasets.

Contribution

It presents a novel Bayesian Gaussian Process framework utilizing Process Kernels for hyperspectral recovery from RGB images, incorporating spectral transformation and clustering techniques.

Findings

Effective spectral detail recovery demonstrated on three datasets.

Outperforms existing methods in spectral accuracy.

Robustness to different spectral quantizations.

Abstract

We propose to recover spectral details from RGB images of known spectral quantization by modeling natural spectra under Gaussian Processes and combining them with the RGB images. Our technique exploits Process Kernels to model the relative smoothness of reflectance spectra, and encourages non-negativity in the resulting signals for better estimation of the reflectance values. The Gaussian Processes are inferred in sets using clusters of spatio-spectrally correlated hyperspectral training patches. Each set is transformed to match the spectral quantization of the test RGB image. We extract overlapping patches from the RGB image and match them to the hyperspectral training patches by spectrally transforming the latter. The RGB patches are encoded over the transformed Gaussian Processes related to those hyperspectral patches and the resulting image is constructed by combining the codes with the original Processes. Our approach infers the desired Gaussian Processes under a fully Bayesian model inspired by Beta-Bernoulli Process, for which we also present the inference procedure. A thorough evaluation using three hyperspectral datasets demonstrates the effective extraction of spectral details from RGB images by the proposed technique.