EventGeM: Global-to-Local Feature Matching for Event-Based Visual Place Recognition

TL;DR

EventGeM introduces a novel event-based visual place recognition pipeline combining global and local features with multi-stage re-ranking, achieving state-of-the-art accuracy and real-time performance across diverse conditions.

Contribution

The paper presents a new global-to-local feature fusion method for event-based place recognition using pre-trained vision transformers and depth models, enhancing accuracy and efficiency.

Findings

Achieves state-of-the-art localization accuracy on benchmark datasets.

Operates in real-time across various hardware architectures.

Effective under diverse lighting conditions and environments.

Abstract

Dynamic vision sensors, also known as event cameras, are rapidly rising in popularity for robotic and computer vision tasks due to their sparse activation and high-temporal resolution. Event cameras have been used in robotic navigation and localization tasks where accurate positioning needs to occur on small and frequent time scales, or when energy concerns are paramount. In this work, we present EventGeM, a state-of-the-art global to local feature fusion pipeline for event-based Visual Place Recognition. We use a pre-trained vision transformer (ViT-S/16) backbone to obtain global feature patch for initial match predictions embeddings from event histogram images. Local feature keypoints were then detected using a pre-trained MaxViT backbone for 2D-homography based re-ranking with RANSAC. For additional re-ranking refinement, we subsequently used a pre-trained vision foundation model for depth estimation to compare structural similarity between references and queries. Our work performs state-of-the-art localization when compared to the best currently available event-based place recognition method across several benchmark datasets and lighting conditions all whilst being fully capable of running in real-time when deployed across a variety of compute architectures. We demonstrate the capability of EventGeM in a real-world deployment on a robotic platform for online localization using event streams directly from an event camera. Project page: https://eventgemvpr.github.io/