Deep Visual Geo-localization Benchmark

TL;DR

This paper introduces an open-source benchmarking framework for visual geo-localization, enabling systematic evaluation of different pipeline components and their impact on performance and system efficiency.

Contribution

It provides a flexible, systematic evaluation protocol and extensive experiments to guide component choices and optimize performance in visual geo-localization tasks.

Findings

Downscaling images to 80% resolution maintains accuracy while reducing extraction time by 36%.

The framework helps identify optimal backbone, aggregation, and negative mining strategies.

Simple engineering techniques can significantly improve geo-localization performance.

Abstract

In this paper, we propose a new open-source benchmarking framework for Visual Geo-localization (VG) that allows to build, train, and test a wide range of commonly used architectures, with the flexibility to change individual components of a geo-localization pipeline. The purpose of this framework is twofold: i) gaining insights into how different components and design choices in a VG pipeline impact the final results, both in terms of performance (recall@N metric) and system requirements (such as execution time and memory consumption); ii) establish a systematic evaluation protocol for comparing different methods. Using the proposed framework, we perform a large suite of experiments which provide criteria for choosing backbone, aggregation and negative mining depending on the use-case and requirements. We also assess the impact of engineering techniques like pre/post-processing, data augmentation and image resizing, showing that better performance can be obtained through somewhat simple procedures: for example, downscaling the images' resolution to 80% can lead to similar results with a 36% savings in extraction time and dataset storage requirement. Code and trained models are available at https://deep-vg-bench.herokuapp.com/.