GEO-Detective: Unveiling Location Privacy Risks in Images with LLM Agents

TL;DR

GEO-Detective is a novel agent that leverages human-like reasoning and external tools to accurately geolocate images shared on social media, revealing significant privacy risks and outperforming existing models especially on images with limited geographic cues.

Contribution

It introduces GEO-Detective, a new approach that adaptively combines reasoning and external clues for improved image geolocation, highlighting privacy concerns and robustness.

Findings

Outperforms baseline LVLMs by over 11.1% at country level geolocation.

Reduces 'unknown' prediction rate by more than 50.6% with external clues.

Demonstrates stronger robustness against privacy defense strategies.

Abstract

Images shared on social media often expose geographic cues. While early geolocation methods required expert effort and lacked generalization, the rise of Large Vision Language Models (LVLMs) now enables accurate geolocation even for ordinary users. However, existing approaches are not optimized for this task. To explore the full potential and associated privacy risks, we present Geo-Detective, an agent that mimics human reasoning and tool use for image geolocation inference. It follows a procedure with four steps that adaptively selects strategies based on image difficulty and is equipped with specialized tools such as visual reverse search, which emulates how humans gather external geographic clues. Experimental results show that GEO-Detective outperforms baseline large vision language models (LVLMs) overall, particularly on images lacking visible geographic features. In country level geolocation tasks, it achieves an improvement of over 11.1% compared to baseline LLMs, and even at finer grained levels, it still provides around a 5.2% performance gain. Meanwhile, when equipped with external clues, GEO-Detective becomes more likely to produce accurate predictions, reducing the "unknown" prediction rate by more than 50.6%. We further explore multiple defense strategies and find that Geo-Detective exhibits stronger robustness, highlighting the need for more effective privacy safeguards.