Non-Uniform Exposure Imaging via Neuromorphic Shutter Control

TL;DR

This paper introduces a neuromorphic shutter control system that uses real-time event data to adapt exposure in cameras, reducing motion blur and noise in challenging environments, supported by a hardware prototype and a new denoising network.

Contribution

It presents a novel neuromorphic shutter control system with real-time adaptive exposure and an event-based denoising network, implemented in hardware for real-world scene imaging.

Findings

Outperforms state-of-the-art methods on synthetic and real datasets.

Successfully stabilizes SNR despite non-uniform exposure times.

Demonstrates real-time adaptive imaging in diverse scenarios.

Abstract

By leveraging the blur-noise trade-off, imaging with non-uniform exposures largely extends the image acquisition flexibility in harsh environments. However, the limitation of conventional cameras in perceiving intra-frame dynamic information prevents existing methods from being implemented in the real-world frame acquisition for real-time adaptive camera shutter control. To address this challenge, we propose a novel Neuromorphic Shutter Control (NSC) system to avoid motion blurs and alleviate instant noises, where the extremely low latency of events is leveraged to monitor the real-time motion and facilitate the scene-adaptive exposure. Furthermore, to stabilize the inconsistent Signal-to-Noise Ratio (SNR) caused by the non-uniform exposure times, we propose an event-based image denoising network within a self-supervised learning paradigm, i.e., SEID, exploring the statistics of image noises and inter-frame motion information of events to obtain artificial supervision signals for high-quality imaging in real-world scenes. To illustrate the effectiveness of the proposed NSC, we implement it in hardware by building a hybrid-camera imaging prototype system, with which we collect a real-world dataset containing well-synchronized frames and events in diverse scenarios with different target scenes and motion patterns. Experiments on the synthetic and real-world datasets demonstrate the superiority of our method over state-of-the-art approaches.