Optical Linear Systems Framework for Event Sensing and Computational Neuromorphic Imaging

TL;DR

This paper introduces a physics-based linear systems framework for processing event-based sensor data, enabling direct inverse filtering and improved imaging in dynamic optical systems.

Contribution

It presents a novel linear systems model for event data, allowing inverse filtering and source localization in neuromorphic imaging.

Findings

Validated in simulation for point sources under defocus

Demonstrated source localization with real event data from a tunable-focus telescope

Enabled separation of overlapping sources using the framework

Abstract

Event vision sensors (neuromorphic cameras) output sparse, asynchronous ON/OFF events triggered by log-intensity threshold crossings, enabling microsecond-scale sensing with high dynamic range and low data bandwidth. As a nonlinear system, this event representation does not readily integrate with the linear forward models that underpin most computational imaging and optical system design. We present a physics-grounded processing pipeline that maps event streams to estimates of per-pixel log-intensity and intensity derivatives, and embeds these measurements in a dynamic linear systems model with a time-varying point spread function. This enables inverse filtering directly from event data, using frequency-domain Wiener deconvolution with a known (or parameterised) dynamic transfer function. We validate the approach in simulation for single and overlapping point sources under modulated defocus, and on real event data from a tunable-focus telescope imaging a star field, demonstrating source localisation and separability. The proposed framework provides a practical bridge between event sensing and model-based computational imaging for dynamic optical systems.