Temporal-Mapping Photography for Event Cameras

TL;DR

This paper introduces a novel method called EvTemMap that converts event camera data into dense intensity images in static scenes using a transmittance adjustment device and a neural network, enabling high dynamic range and detailed imaging.

Contribution

The paper presents the first approach for static scene event-to-image conversion using a transmittance adjustment device and temporal mapping neural network, expanding event camera applications.

Findings

High dynamic range imaging achieved

Fine-grained detail preservation demonstrated

Effective in low-light and high dynamic range scenes

Abstract

Event cameras, or Dynamic Vision Sensors (DVS) are novel neuromorphic sensors that capture brightness changes as a continuous stream of "events" rather than traditional intensity frames. Converting sparse events to dense intensity frames faithfully has long been an ill-posed problem. Previous methods have primarily focused on converting events to video in dynamic scenes or with a moving camera. In this paper, for the first time, we realize events to dense intensity image conversion using a stationary event camera in static scenes with a transmittance adjustment device for brightness modulation. Different from traditional methods that mainly rely on event integration, the proposed Event-Based Temporal Mapping Photography (EvTemMap) measures the time of event emitting for each pixel. Then, the resulting Temporal Matrix is converted to an intensity frame with a temporal mapping neural network. At the hardware level, the proposed EvTemMap is implemented by combining a transmittance adjustment device with a DVS, named Adjustable Transmittance Dynamic Vision Sensor (AT-DVS). Additionally, we collected TemMat dataset under various conditions including low-light and high dynamic range scenes. The experimental results showcase the high dynamic range, fine-grained details, and high-grayscale resolution of the proposed EvTemMap. The code and dataset are available in https://github.com/YuHanBaozju/EvTemMap