Tuning IR-cut Filter for Illumination-aware Spectral Reconstruction from RGB

TL;DR

This paper proposes tuning the IR-cut filter in RGB cameras combined with a deep learning approach to improve spectral reconstruction accuracy and also recover illumination spectra, balancing performance and complexity.

Contribution

It introduces a novel IR-cut filter tuning strategy and a deep learning-based spectral reconstruction method that jointly estimate illumination spectra from RGB images.

Findings

Enhanced spectral reconstruction accuracy with tuned IR-cut filters.

Effective illumination spectrum recovery demonstrated on synthetic and real images.

Better trade-off between accuracy and implementation complexity achieved.

Abstract

To reconstruct spectral signals from multi-channel observations, in particular trichromatic RGBs, has recently emerged as a promising alternative to traditional scanning-based spectral imager. It has been proven that the reconstruction accuracy relies heavily on the spectral response of the RGB camera in use. To improve accuracy, data-driven algorithms have been proposed to retrieve the best response curves of existing RGB cameras, or even to design brand new three-channel response curves. Instead, this paper explores the filter-array based color imaging mechanism of existing RGB cameras, and proposes to design the IR-cut filter properly for improved spectral recovery, which stands out as an in-between solution with better trade-off between reconstruction accuracy and implementation complexity. We further propose a deep learning based spectral reconstruction method, which allows to recover the illumination spectrum as well. Experiment results with both synthetic and real images under daylight illumination have shown the benefits of our IR-cut filter tuning method and our illumination-aware spectral reconstruction method.