Image-Based Geolocation Using Large Vision-Language Models

TL;DR

This paper introduces ool{}, a novel framework leveraging large vision-language models and a chain-of-thought approach to significantly improve image-based geolocation accuracy, outperforming traditional methods and human benchmarks while addressing privacy concerns.

Contribution

The paper presents ool{}, a new framework that enhances geolocation accuracy using LVLMs and a systematic reasoning approach, addressing privacy risks and dataset issues.

Findings

ool{} achieves an average score of 4550.5 in GeoGuessr.

ool{} attains an 85.37% win rate in geolocation tasks.

Closest predictions are accurate within 0.3 km.

Abstract

Geolocation is now a vital aspect of modern life, offering numerous benefits but also presenting serious privacy concerns. The advent of large vision-language models (LVLMs) with advanced image-processing capabilities introduces new risks, as these models can inadvertently reveal sensitive geolocation information. This paper presents the first in-depth study analyzing the challenges posed by traditional deep learning and LVLM-based geolocation methods. Our findings reveal that LVLMs can accurately determine geolocations from images, even without explicit geographic training. To address these challenges, we introduce \tool{}, an innovative framework that significantly enhances image-based geolocation accuracy. \tool{} employs a systematic chain-of-thought (CoT) approach, mimicking human geoguessing strategies by carefully analyzing visual and contextual cues such as vehicle types, architectural styles, natural landscapes, and cultural elements. Extensive testing on a dataset of 50,000 ground-truth data points shows that \tool{} outperforms both traditional models and human benchmarks in accuracy. It achieves an impressive average score of 4550.5 in the GeoGuessr game, with an 85.37\% win rate, and delivers highly precise geolocation predictions, with the closest distances as accurate as 0.3 km. Furthermore, our study highlights issues related to dataset integrity, leading to the creation of a more robust dataset and a refined framework that leverages LVLMs' cognitive capabilities to improve geolocation precision. These findings underscore \tool{}'s superior ability to interpret complex visual data, the urgent need to address emerging security vulnerabilities posed by LVLMs, and the importance of responsible AI development to ensure user privacy protection.