EPRBench: A High-Quality Benchmark Dataset for Event Stream Based Visual Place Recognition

TL;DR

EPRBench is a new high-quality dataset for event stream-based visual place recognition, supporting advanced research with diverse real-world data, semantic descriptions, and a novel multi-modal fusion approach that improves accuracy and interpretability.

Contribution

The paper introduces EPRBench, a comprehensive dataset for event-based VPR, along with a novel multi-modal fusion framework leveraging LLMs for enhanced accuracy and explainability.

Findings

Benchmarking 15 state-of-the-art algorithms on EPRBench

Proposed multi-modal fusion method improves recognition accuracy

Framework provides interpretable reasoning alongside predictions

Abstract

Event stream-based Visual Place Recognition (VPR) is an emerging research direction that offers a compelling solution to the instability of conventional visible-light cameras under challenging conditions such as low illumination, overexposure, and high-speed motion. Recognizing the current scarcity of dedicated datasets in this domain, we introduce EPRBench, a high-quality benchmark specifically designed for event stream-based VPR. EPRBench comprises 10K event sequences and 65K event frames, collected using both handheld and vehicle-mounted setups to comprehensively capture real-world challenges across diverse viewpoints, weather conditions, and lighting scenarios. To support semantic-aware and language-integrated VPR research, we provide LLM-generated scene descriptions, subsequently refined through human annotation, establishing a solid foundation for integrating LLMs into event-based perception pipelines. To facilitate systematic evaluation, we implement and benchmark 15 state-of-the-art VPR algorithms on EPRBench, offering a strong baseline for future algorithmic comparisons. Furthermore, we propose a novel multi-modal fusion paradigm for VPR: leveraging LLMs to generate textual scene descriptions from raw event streams, which then guide spatially attentive token selection, cross-modal feature fusion, and multi-scale representation learning. This framework not only achieves highly accurate place recognition but also produces interpretable reasoning processes alongside its predictions, significantly enhancing model transparency and explainability. The dataset and source code will be released on https://github.com/Event-AHU/Neuromorphic_ReID