Learning to See Through Flare

TL;DR

NeuSee is a novel computational imaging framework that protects sensors from laser flare by jointly learning a diffractive optical element and a neural network for high-fidelity image restoration, outperforming previous methods.

Contribution

It introduces the first full-spectrum sensor protection system combining a learned DOE and neural networks, capable of suppressing intense laser flare in diverse real-world conditions.

Findings

Suppresses laser irradiance up to 10^6 times I_sat

Achieves 10.1% improvement in image quality over prior methods

Enables full-spectrum imaging and flare suppression in complex scenes

Abstract

Machine vision systems are susceptible to laser flare, where unwanted intense laser illumination blinds and distorts its perception of the environment through oversaturation or permanent damage to sensor pixels. We introduce NeuSee, the first computational imaging framework for high-fidelity sensor protection across the full visible spectrum. It jointly learns a neural representation of a diffractive optical element (DOE) and a frequency-space Mamba-GAN network for image restoration. NeuSee system is adversarially trained end-to-end on 100K unique images to suppress the peak laser irradiance as high as $10^6$ times the sensor saturation threshold $I_{\textrm{sat}}$, the point at which camera sensors may experience damage without the DOE. Our system leverages heterogeneous data and model parallelism for distributed computing, integrating hyperspectral information and multiple neural networks for realistic simulation and image restoration. NeuSee takes into account open-world scenes with dynamically varying laser wavelengths, intensities, and positions, as well as lens flare effects, unknown ambient lighting conditions, and sensor noises. It outperforms other learned DOEs, achieving full-spectrum imaging and laser suppression for the first time, with a 10.1\% improvement in restored image quality.