AmsterTime: A Visual Place Recognition Benchmark Dataset for Severe Domain Shift

TL;DR

AmsterTime presents a challenging, crowdsourced dataset for evaluating visual place recognition under severe domain shifts, with comprehensive benchmarks and analysis of various models and explainability techniques.

Contribution

The paper introduces AmsterTime, a novel large-scale dataset for VPR under severe domain shifts, and provides extensive benchmark evaluations and interpretability analyses.

Findings

ResNet-101 pre-trained on Landmarks dataset achieves 84% verification accuracy.

The dataset enables evaluation of models on verification and retrieval tasks.

Grad-CAM visualizations help interpret learned features in deep metric models.

Abstract

We introduce AmsterTime: a challenging dataset to benchmark visual place recognition (VPR) in presence of a severe domain shift. AmsterTime offers a collection of 2,500 well-curated images matching the same scene from a street view matched to historical archival image data from Amsterdam city. The image pairs capture the same place with different cameras, viewpoints, and appearances. Unlike existing benchmark datasets, AmsterTime is directly crowdsourced in a GIS navigation platform (Mapillary). We evaluate various baselines, including non-learning, supervised and self-supervised methods, pre-trained on different relevant datasets, for both verification and retrieval tasks. Our result credits the best accuracy to the ResNet-101 model pre-trained on the Landmarks dataset for both verification and retrieval tasks by 84% and 24%, respectively. Additionally, a subset of Amsterdam landmarks is collected for feature evaluation in a classification task. Classification labels are further used to extract the visual explanations using Grad-CAM for inspection of the learned similar visuals in a deep metric learning models.