Object-Level Explanations for Image Geolocation Models: a GeoGuessr use-case

TL;DR

This paper introduces an object-centric analysis pipeline to interpret image geolocation models, demonstrating that attribution maps can be decomposed into meaningful, object-like visual elements that influence model predictions.

Contribution

It presents a novel method to extract and evaluate object-like regions from attribution maps, enabling object-level interpretability of geolocation models.

Findings

Attribution-guided crops retain more predictive information than random regions.

Decomposing attribution maps into objects enhances interpretability.

Experiments on a three-country benchmark validate the approach.

Abstract

When humans play geolocation games such as GeoGuessr, they rely on concrete visual cues, such as road markings, vegetation, or architectural details, to infer where an image was captured. Whether image geolocation models rely on similar object-level evidence remains difficult to determine, as attribution methods like Grad-CAM typically highlight diffuse regions rather than coherent visual entities, making it difficult to link model predictions to specific objects or perceptible patterns. In this work, we propose an object-centric analysis pipeline to investigate the visual evidence used by geolocation models. Starting from attribution maps, we extract salient regions and segment them into object-like elements. We evaluate their predictive relevance through deletion and insertion tests, comparing attributionguided crops to randomly selected regions with similar coverage. Experiments on a three-country benchmark show that attribution-guided crops consistently retain more information for the model's prediction than random crops. These results suggest that attribution maps can be decomposed into interpretable, perceptible elements, providing a step toward object-level analysis of geolocation models.