BlinkTrack: Feature Tracking over 80 FPS via Events and Images
Yichen Shen, Yijin Li, Shuo Chen, Guanglin Li, Zhaoyang Huang, Hujun Bao, Zhaopeng Cui, Guofeng Zhang

TL;DR
BlinkTrack is a novel high-frequency feature tracking framework that combines event camera data with images, extending Kalman filters into a learning-based approach to achieve over 80 FPS in challenging conditions.
Contribution
The paper introduces a differentiable Kalman filter-based framework that fuses event and image data for improved feature tracking, along with new datasets for evaluation.
Findings
Achieves over 80 FPS with multi-modality data
Outperforms existing methods in accuracy and speed
Effectively fuses asynchronous event and image data
Abstract
Event cameras, known for their high temporal resolution and ability to capture asynchronous changes, have gained significant attention for their potential in feature tracking, especially in challenging conditions. However, event cameras lack the fine-grained texture information that conventional cameras provide, leading to error accumulation in tracking. To address this, we propose a novel framework, BlinkTrack, which integrates event data with grayscale images for high-frequency feature tracking. Our method extends the traditional Kalman filter into a learning-based framework, utilizing differentiable Kalman filters in both event and image branches. This approach improves single-modality tracking and effectively solves the data association and fusion from asynchronous event and image data. We also introduce new synthetic and augmented datasets to better evaluate our model. Experimental…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsWeb Data Mining and Analysis · Topic Modeling · Machine Learning and Data Classification
MethodsSoftmax · Attention Is All You Need
