TMA: Temporal Motion Aggregation for Event-based Optical Flow

TL;DR

This paper introduces TMA, a novel approach for event-based optical flow that leverages temporal continuity through motion aggregation, resulting in more accurate and efficient flow estimation compared to existing methods.

Contribution

The paper proposes a new Temporal Motion Aggregation (TMA) method that effectively incorporates temporal information for improved event-based optical flow estimation.

Findings

TMA outperforms existing methods in accuracy on DSEC-Flow and MVSEC datasets.

TMA reduces inference time by 40% compared to E-RAFT.

TMA achieves a 6% accuracy improvement over E-RAFT.

Abstract

Event cameras have the ability to record continuous and detailed trajectories of objects with high temporal resolution, thereby providing intuitive motion cues for optical flow estimation. Nevertheless, most existing learning-based approaches for event optical flow estimation directly remould the paradigm of conventional images by representing the consecutive event stream as static frames, ignoring the inherent temporal continuity of event data. In this paper, we argue that temporal continuity is a vital element of event-based optical flow and propose a novel Temporal Motion Aggregation (TMA) approach to unlock its potential. Technically, TMA comprises three components: an event splitting strategy to incorporate intermediate motion information underlying the temporal context, a linear lookup strategy to align temporally fine-grained motion features and a novel motion pattern aggregation module to emphasize consistent patterns for motion feature enhancement. By incorporating temporally fine-grained motion information, TMA can derive better flow estimates than existing methods at early stages, which not only enables TMA to obtain more accurate final predictions, but also greatly reduces the demand for a number of refinements. Extensive experiments on DSEC-Flow and MVSEC datasets verify the effectiveness and superiority of our TMA. Remarkably, compared to E-RAFT, TMA achieves a 6\% improvement in accuracy and a 40\% reduction in inference time on DSEC-Flow. Code will be available at \url{https://github.com/ispc-lab/TMA}.