Snapshot multi-spectral imaging through defocusing and a Fourier imager network

TL;DR

This paper presents a novel snapshot multi-spectral imaging method that uses defocusing and a deep learning Fourier network to reconstruct spectral information from a single monochrome image, eliminating the need for filters.

Contribution

The work introduces a physical encoding via chromatic aberration combined with a deep learning decoder for multi-spectral imaging using standard sensors, which is a new approach.

Findings

Achieved 92.98% accuracy in spectral channel prediction.

Demonstrated high-quality multi-spectral reconstruction on various objects.

Enabled multi-spectral imaging without additional filters or specialized hardware.

Abstract

Multi-spectral imaging, which simultaneously captures the spatial and spectral information of a scene, is widely used across diverse fields, including remote sensing, biomedical imaging, and agricultural monitoring. Here, we introduce a snapshot multi-spectral imaging approach employing a standard monochrome image sensor with no additional spectral filters or customized components. Our system leverages the inherent chromatic aberration of wavelength-dependent defocusing as a natural source of physical encoding of multi-spectral information; this encoded image information is rapidly decoded via a deep learning-based multi-spectral Fourier Imager Network (mFIN). We experimentally tested our method with six illumination bands and demonstrated an overall accuracy of 92.98% for predicting the illumination channels at the input and achieved a robust multi-spectral image reconstruction on various test objects. This deep learning-powered framework achieves high-quality multi-spectral image reconstruction using snapshot image acquisition with a monochrome image sensor and could be useful for applications in biomedicine, industrial quality control, and agriculture, among others.