MMS-VPR: Multimodal Street-Level Visual Place Recognition Dataset and Benchmark

TL;DR

MMS-VPR introduces a comprehensive multimodal street-level dataset and benchmark for visual place recognition, emphasizing pedestrian environments, diverse modalities, and long-term, day-night, and multi-angle data in a non-Western urban setting.

Contribution

The paper presents MMS-VPR, a large-scale multimodal dataset and benchmarking platform specifically designed for pedestrian street-level visual place recognition, addressing limitations of existing vehicle-centric datasets.

Findings

Provides 110,529 images and 2,527 video clips across 208 locations.

Includes multimodal annotations like GPS, timestamps, and textual metadata.

Offers a standardized benchmarking platform for multimodal VPR methods.

Abstract

Existing visual place recognition (VPR) datasets predominantly rely on vehicle-mounted imagery, offer limited multimodal diversity, and underrepresent dense pedestrian street scenes, particularly in non-Western urban contexts. We introduce MMS-VPR, a large-scale multimodal dataset for street-level place recognition in pedestrian-only environments. MMS-VPR comprises 110,529 images and 2,527 video clips across 208 locations in a ~70,800 $m^2$ open-air commercial district in Chengdu, China. Field data were collected in 2024, while social media data span seven years (2019-2025), providing both fine-grained temporal granularity and long-term temporal coverage. Each location features comprehensive day-night coverage, multiple viewing angles, and multimodal annotations including GPS coordinates, timestamps, and semantic textual metadata. We further release MMS-VPRlib, a unified benchmarking platform that consolidates commonly used VPR datasets and state-of-the-art methods under a standardized, reproducible pipeline. MMS-VPRlib provides modular components for data pre-processing, multimodal modeling (CNN/RNN/Transformer), signal enhancement, alignment, fusion, and performance evaluation. This platform moves beyond traditional image-only paradigms, enabling systematic exploitation of complementary visual, video, and textual modalities. The dataset is available at https://huggingface.co/datasets/Yiwei-Ou/MMS-VPR and the benchmark at https://github.com/yiasun/MMS-VPRlib.