End-to-end joint optimization of metasurface and image processing for compact snapshot hyperspectral imaging

TL;DR

This paper introduces a compact hyperspectral imaging system that jointly optimizes a metasurface and neural network processing, enabling high-quality spectral reconstruction in a smaller, portable form factor.

Contribution

It presents a fully differentiable model for joint optimization of metasurface design and image processing, advancing miniaturization of hyperspectral imaging technology.

Findings

Outperforms traditional separation optimization methods in spectral reconstruction quality.

Exploits metasurface dispersion for spectral differentiation.

Demonstrates potential for miniaturized, portable hyperspectral systems.

Abstract

Traditional snapshot hyperspectral imaging systems generally require multiple refractive-optics-based elements to modulate light, resulting in bulky framework. In pursuit of a more compact form factor, a metasurface-based snapshot hyperspectral imaging system, which achieves joint optimization of metasurface and image processing, is proposed in this paper. The unprecedented light manipulation capabilities of metasurfaces are used in conjunction with neural networks to encode and decode light fields for better hyperspectral imaging. Specifically, the extremely strong dispersion of metasurfaces is exploited to distinguish spectral information, and a neural network based on spectral priors is applied for hyperspectral image reconstruction. By constructing a fully differentiable model of metasurface-based hyperspectral imaging, the front-end metasurface phase distribution and the back-end recovery network parameters can be jointly optimized. This method achieves high-quality hyperspectral reconstruction results numerically, outperforming separation optimization methods. The proposed system holds great potential for miniaturization and portability of hyperspectral imaging systems.