Cross-modal Place Recognition in Image Databases using Event-based Sensors

TL;DR

This paper introduces a novel cross-modal visual place recognition framework that retrieves regular images using event-based sensors, improving robustness in challenging lighting conditions and outperforming existing methods.

Contribution

It is the first to enable cross-modal retrieval from event sensors to frame images, combining retrieval and classification for enhanced accuracy.

Findings

Outperforms state-of-the-art frame-based and event-based methods.

Effective in difficult illumination and appearance change scenarios.

Combining retrieval and classification significantly boosts performance.

Abstract

Visual place recognition is an important problem towards global localization in many robotics tasks. One of the biggest challenges is that it may suffer from illumination or appearance changes in surrounding environments. Event cameras are interesting alternatives to frame-based sensors as their high dynamic range enables robust perception in difficult illumination conditions. However, current event-based place recognition methods only rely on event information, which restricts downstream applications of VPR. In this paper, we present the first cross-modal visual place recognition framework that is capable of retrieving regular images from a database given an event query. Our method demonstrates promising results with respect to the state-of-the-art frame-based and event-based methods on the Brisbane-Event-VPR dataset under different scenarios. We also verify the effectiveness of the combination of retrieval and classification, which can boost performance by a large margin.