ETAP: Event-based Tracking of Any Point
Friedhelm Hamann, Daniel Gehrig, Filbert Febryanto, Kostas Daniilidis,, Guillermo Gallego

TL;DR
ETAP introduces a novel event camera-based method for tracking any point that leverages high temporal resolution and global context to achieve robust high-speed tracking, outperforming existing approaches significantly.
Contribution
This work is the first to develop an event camera-based TAP method, incorporating a feature-alignment loss for motion-robust features and demonstrating strong generalization and performance improvements.
Findings
136% better on average Jaccard metric than baselines
20% improvement on feature tracking benchmark over previous event-only method
Surpasses previous best events-and-frames method by 4.1%
Abstract
Tracking any point (TAP) recently shifted the motion estimation paradigm from focusing on individual salient points with local templates to tracking arbitrary points with global image contexts. However, while research has mostly focused on driving the accuracy of models in nominal settings, addressing scenarios with difficult lighting conditions and high-speed motions remains out of reach due to the limitations of the sensor. This work addresses this challenge with the first event camera-based TAP method. It leverages the high temporal resolution and high dynamic range of event cameras for robust high-speed tracking, and the global contexts in TAP methods to handle asynchronous and sparse event measurements. We further extend the TAP framework to handle event feature variations induced by motion -- thereby addressing an open challenge in purely event-based tracking -- with a novel…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTarget Tracking and Data Fusion in Sensor Networks · Infrared Target Detection Methodologies
