Time-Ordered Recent Event (TORE) Volumes for Event Cameras

TL;DR

The paper introduces TORE volumes, a novel event representation for event cameras that efficiently preserves raw timing information, improving performance across various tasks without relying on interpolated data.

Contribution

It presents TORE volumes, a new bio-inspired, memory-efficient event representation that enhances machine learning performance on event camera data.

Findings

Significantly improves state-of-the-art results in event denoising.

Enhances image reconstruction and classification accuracy.

Facilitates human pose estimation with higher precision.

Abstract

Event cameras are an exciting, new sensor modality enabling high-speed imaging with extremely low-latency and wide dynamic range. Unfortunately, most machine learning architectures are not designed to directly handle sparse data, like that generated from event cameras. Many state-of-the-art algorithms for event cameras rely on interpolated event representations - obscuring crucial timing information, increasing the data volume, and limiting overall network performance. This paper details an event representation called Time-Ordered Recent Event (TORE) volumes. TORE volumes are designed to compactly store raw spike timing information with minimal information loss. This bio-inspired design is memory efficient, computationally fast, avoids time-blocking (i.e. fixed and predefined frame rates), and contains "local memory" from past data. The design is evaluated on a wide range of challenging tasks (e.g. event denoising, image reconstruction, classification, and human pose estimation) and is shown to dramatically improve state-of-the-art performance. TORE volumes are an easy-to-implement replacement for any algorithm currently utilizing event representations.