MeshVPR: Citywide Visual Place Recognition Using 3D Meshes

TL;DR

MeshVPR introduces a scalable citywide visual place recognition system using 3D meshes, addressing domain gaps with a features alignment framework, and demonstrates competitive results on new datasets from Berlin, Paris, and Melbourne.

Contribution

The paper presents MeshVPR, a novel pipeline that effectively uses synthetic 3D meshes for large-scale visual place recognition, bridging the domain gap with a lightweight features alignment method.

Findings

MeshVPR achieves competitive performance with standard VPR methods.

Synthetic mesh-based databases initially underperform compared to real images.

The proposed alignment framework significantly improves retrieval accuracy.

Abstract

Mesh-based scene representation offers a promising direction for simplifying large-scale hierarchical visual localization pipelines, combining a visual place recognition step based on global features (retrieval) and a visual localization step based on local features. While existing work demonstrates the viability of meshes for visual localization, the impact of using synthetic databases rendered from them in visual place recognition remains largely unexplored. In this work we investigate using dense 3D textured meshes for large-scale Visual Place Recognition (VPR). We identify a significant performance drop when using synthetic mesh-based image databases compared to real-world images for retrieval. To address this, we propose MeshVPR, a novel VPR pipeline that utilizes a lightweight features alignment framework to bridge the gap between real-world and synthetic domains. MeshVPR leverages pre-trained VPR models and is efficient and scalable for city-wide deployments. We introduce novel datasets with freely available 3D meshes and manually collected queries from Berlin, Paris, and Melbourne. Extensive evaluations demonstrate that MeshVPR achieves competitive performance with standard VPR pipelines, paving the way for mesh-based localization systems. Data, code, and interactive visualizations are available at https://meshvpr.github.io/