CodedEvents: Optimal Point-Spread-Function Engineering for 3D-Tracking with Event Cameras

TL;DR

This paper introduces a theoretical framework and novel mask designs for 3D tracking of moving point sources using event cameras, enhancing localization accuracy beyond existing methods.

Contribution

It establishes fundamental limits for event camera PSF engineering and develops optimal phase and amplitude masks for 3D tracking of moving sources.

Findings

Existing Fisher phase masks are near-optimal for static sources.

Designed new masks outperform existing ones for moving source tracking.

Validated designs through simulations and a prototype.

Abstract

Point-spread-function (PSF) engineering is a well-established computational imaging technique that uses phase masks and other optical elements to embed extra information (e.g., depth) into the images captured by conventional CMOS image sensors. To date, however, PSF-engineering has not been applied to neuromorphic event cameras; a powerful new image sensing technology that responds to changes in the log-intensity of light. This paper establishes theoretical limits (Cram\'er Rao bounds) on 3D point localization and tracking with PSF-engineered event cameras. Using these bounds, we first demonstrate that existing Fisher phase masks are already near-optimal for localizing static flashing point sources (e.g., blinking fluorescent molecules). We then demonstrate that existing designs are sub-optimal for tracking moving point sources and proceed to use our theory to design optimal phase masks and binary amplitude masks for this task. To overcome the non-convexity of the design problem, we leverage novel implicit neural representation based parameterizations of the phase and amplitude masks. We demonstrate the efficacy of our designs through extensive simulations. We also validate our method with a simple prototype.