VLM-Guided Visual Place Recognition for Planet-Scale Geo-Localization

TL;DR

This paper introduces a hybrid geo-localization system that combines vision-language models with retrieval-based visual place recognition, significantly improving accuracy and scalability for planet-scale image localization tasks.

Contribution

It proposes a novel framework that uses VLMs to guide and constrain visual place recognition, enhancing robustness and performance over prior methods.

Findings

Outperforms state-of-the-art methods on multiple benchmarks.

Achieves up to 4.51% improvement at street level.

Achieves up to 13.52% improvement at city level.

Abstract

Geo-localization from a single image at planet scale (essentially an advanced or extreme version of the kidnapped robot problem) is a fundamental and challenging task in applications such as navigation, autonomous driving and disaster response due to the vast diversity of locations, environmental conditions, and scene variations. Traditional retrieval-based methods for geo-localization struggle with scalability and perceptual aliasing, while classification-based approaches lack generalization and require extensive training data. Recent advances in vision-language models (VLMs) offer a promising alternative by leveraging contextual understanding and reasoning. However, while VLMs achieve high accuracy, they are often prone to hallucinations and lack interpretability, making them unreliable as standalone solutions. In this work, we propose a novel hybrid geo-localization framework that combines the strengths of VLMs with retrieval-based visual place recognition (VPR) methods. Our approach first leverages a VLM to generate a prior, effectively guiding and constraining the retrieval search space. We then employ a retrieval step, followed by a re-ranking mechanism that selects the most geographically plausible matches based on feature similarity and proximity to the initially estimated coordinates. We evaluate our approach on multiple geo-localization benchmarks and show that it consistently outperforms prior state-of-the-art methods, particularly at street (up to 4.51%) and city level (up to 13.52%). Our results demonstrate that VLM-generated geographic priors in combination with VPR lead to scalable, robust, and accurate geo-localization systems.