Hyperspectral Light Field Stereo Matching

TL;DR

This paper introduces a novel hyperspectral light field system and a cross-spectral stereo matching technique that leverages spectral-invariant features and MRF optimization to accurately estimate scene depth and generate high-quality disparity maps.

Contribution

It presents a new hyperspectral light field capture system and a spectral-invariant feature descriptor with a matching metric, advancing depth estimation in hyperspectral imaging.

Findings

High-quality disparity maps from synthetic and real data

Effective synthesis of all-focus and defocused images

Demonstrated robustness of the method across spectral variations

Abstract

In this paper, we describe how scene depth can be extracted using a hyperspectral light field capture (H-LF) system. Our H-LF system consists of a 5 x 6 array of cameras, with each camera sampling a different narrow band in the visible spectrum. There are two parts to extracting scene depth. The first part is our novel cross-spectral pairwise matching technique, which involves a new spectral-invariant feature descriptor and its companion matching metric we call bidirectional weighted normalized cross correlation (BWNCC). The second part, namely, H-LF stereo matching, uses a combination of spectral-dependent correspondence and defocus cues that rely on BWNCC. These two new cost terms are integrated into a Markov Random Field (MRF) for disparity estimation. Experiments on synthetic and real H-LF data show that our approach can produce high-quality disparity maps. We also show that these results can be used to produce the complete plenoptic cube in addition to synthesizing all-focus and defocused color images under different sensor spectral responses.