Generalized Event Cameras

TL;DR

This paper introduces generalized event cameras using single-photon sensors to preserve scene intensity efficiently, enabling high-speed, high-fidelity imaging with minimal bandwidth and supporting versatile downstream tasks.

Contribution

It proposes a novel framework for generalized event cameras that encode richer scene information and demonstrates their implementation using single-photon sensors for improved performance.

Findings

High-speed, high-fidelity imaging at low readout rates

Supports plug-and-play downstream inference without specialized datasets

Bandwidth-efficient preservation of scene intensity

Abstract

Event cameras capture the world at high time resolution and with minimal bandwidth requirements. However, event streams, which only encode changes in brightness, do not contain sufficient scene information to support a wide variety of downstream tasks. In this work, we design generalized event cameras that inherently preserve scene intensity in a bandwidth-efficient manner. We generalize event cameras in terms of when an event is generated and what information is transmitted. To implement our designs, we turn to single-photon sensors that provide digital access to individual photon detections; this modality gives us the flexibility to realize a rich space of generalized event cameras. Our single-photon event cameras are capable of high-speed, high-fidelity imaging at low readout rates. Consequently, these event cameras can support plug-and-play downstream inference, without capturing new event datasets or designing specialized event-vision models. As a practical implication, our designs, which involve lightweight and near-sensor-compatible computations, provide a way to use single-photon sensors without exorbitant bandwidth costs.