MATE: Motion-Augmented Temporal Consistency for Event-based Point Tracking

TL;DR

This paper introduces MATE, an event-based framework that improves point tracking accuracy by addressing spatial sparsity and motion sensitivity using motion-guidance and variable motion modules, validated on new datasets.

Contribution

The paper presents a novel event-based tracking method with modules that handle sparsity and motion variability, outperforming existing approaches on multiple benchmarks.

Findings

Improves $Survival_{50}$ by 17.9% over baseline.

Outperforms all existing event and video-based tracking methods.

Validated on two new simulation datasets.

Abstract

Tracking Any Point (TAP) plays a crucial role in motion analysis. Video-based approaches rely on iterative local matching for tracking, but they assume linear motion during the blind time between frames, which leads to point loss under large displacements or nonlinear motion. The high temporal resolution and motion blur-free characteristics of event cameras provide continuous, fine-grained motion information, capturing subtle variations with microsecond precision. This paper presents an event-based framework for tracking any point, which tackles the challenges posed by spatial sparsity and motion sensitivity in events through two tailored modules. Specifically, to resolve ambiguities caused by event sparsity, a motion-guidance module incorporates kinematic vectors into the local matching process. Additionally, a variable motion aware module is integrated to ensure temporally consistent responses that are insensitive to varying velocities, thereby enhancing matching precision. To validate the effectiveness of the approach, two event dataset for tracking any point is constructed by simulation. The method improves the $Survival_{50}$ metric by 17.9% over event-only tracking of any point baseline. Moreover, on standard feature tracking benchmarks, it outperforms all existing methods, even those that combine events and video frames.