Foveated Thermal Computational Imaging in the Wild Using All-Silicon Meta-Optics

TL;DR

This paper introduces a novel all-silicon meta-optical system for real-time, foveated thermal imaging that combines polarization-sensitive optics with computational reconstruction to achieve high-resolution focus in the center and wide field of view.

Contribution

It presents the first all-silicon, polarization-sensitive meta-optical design for foveated thermal imaging, integrating hardware and deep learning for real-time in-the-wild applications.

Findings

Achieved 12 fps real-time thermal foveated imaging.

Demonstrated large-aperture, polarization-sensitive meta-surfaces.

Successfully reconstructed high-resolution foveal images with wide context.

Abstract

Foveated imaging provides a better tradeoff between situational awareness (field of view) and resolution and is critical in long-wavelength infrared regimes because of the size, weight, power, and cost of thermal sensors. We demonstrate computational foveated imaging by exploiting the ability of a meta-optical frontend to discriminate between different polarization states and a computational backend to reconstruct the captured image/video. The frontend is a three-element optic: the first element which we call the "foveal" element is a metalens that focuses s-polarized light at a distance of $f_1$ without affecting the p-polarized light; the second element which we call the "perifoveal" element is another metalens that focuses p-polarized light at a distance of $f_2$ without affecting the s-polarized light. The third element is a freely rotating polarizer that dynamically changes the mixing ratios between the two polarization states. Both the foveal element (focal length = 150mm; diameter = 75mm), and the perifoveal element (focal length = 25mm; diameter = 25mm) were fabricated as polarization-sensitive, all-silicon, meta surfaces resulting in a large-aperture, 1:6 foveal expansion, thermal imaging capability. A computational backend then utilizes a deep image prior to separate the resultant multiplexed image or video into a foveated image consisting of a high-resolution center and a lower-resolution large field of view context. We build a first-of-its-kind prototype system and demonstrate 12 frames per second real-time, thermal, foveated image, and video capture in the wild.