Labits: Layered Bidirectional Time Surfaces Representation for Event Camera-based Continuous Dense Trajectory Estimation

TL;DR

Labits introduces a layered bidirectional time surface representation for event cameras that preserves detailed temporal and visual information, significantly improving dense trajectory estimation accuracy.

Contribution

The paper proposes Labits, a novel layered bidirectional time surface representation, and a module for extracting local optical flow, enhancing event-based trajectory estimation.

Findings

Achieves 49% reduction in trajectory end-point error (TEPE) over previous methods.

Effectively preserves fine-grained temporal and visual features in event data.

Demonstrates superior performance on the MultiFlow dataset.

Abstract

Event cameras provide a compelling alternative to traditional frame-based sensors, capturing dynamic scenes with high temporal resolution and low latency. Moving objects trigger events with precise timestamps along their trajectory, enabling smooth continuous-time estimation. However, few works have attempted to optimize the information loss during event representation construction, imposing a ceiling on this task. Fully exploiting event cameras requires representations that simultaneously preserve fine-grained temporal information, stable and characteristic 2D visual features, and temporally consistent information density, an unmet challenge in existing representations. We introduce Labits: Layered Bidirectional Time Surfaces, a simple yet elegant representation designed to retain all these features. Additionally, we propose a dedicated module for extracting active pixel local optical flow (APLOF), significantly boosting the performance. Our approach achieves an impressive 49% reduction in trajectory end-point error (TEPE) compared to the previous state-of-the-art on the MultiFlow dataset. The code will be released upon acceptance.