Numerical Demultiplexing of Color Image Sensor Measurements via Non-linear Random Forest Modeling

TL;DR

This paper presents a novel numerical framework using non-linear random forest modeling to perform multispectral imaging with conventional color sensors, enabling spectral demultiplexing without specialized hardware.

Contribution

It introduces a comprehensive numerical demultiplexing approach based on a forward sensor model and machine learning, allowing multispectral imaging with standard sensors.

Findings

Effective spectral demultiplexing demonstrated in simulations

Real-world experiments validate the method's accuracy

Enables higher resolution reflectance spectra from standard sensors

Abstract

The simultaneous capture of imaging data at multiple wavelengths across the electromagnetic spectrum is highly challenging, requiring complex and costly multispectral image sensors. In this study, we introduce a comprehensive framework for performing simultaneous multispectral imaging using conventional image sensors with color filter arrays via numerical demultiplexing of the color image sensor measurements. A numerical forward model characterizing the formation of sensor measurements from light spectra hitting the sensor is constructed based on a comprehensive spectral characterization of the sensor. A numerical demultiplexer is then learned via non-linear random forest modeling based on the forward model. Given the learned numerical demultiplexer, one can then demultiplex simultaneously-acquired measurements made by the image sensor into reflectance intensities at discrete selectable wavelengths, resulting in a higher resolution reflectance spectrum. Simulation and real-world experimental results demonstrate the efficacy of such a method for simultaneous multispectral imaging.