# Assessing the Geolocation Capabilities, Limitations and Societal Risks of Generative Vision-Language Models

## TL;DR

This paper evaluates the geolocation accuracy of 25 state-of-the-art Vision-Language Models across diverse datasets, revealing strengths, limitations, and societal privacy risks associated with their use.

## Contribution

It provides a systematic assessment of VLMs' geolocation capabilities, highlighting their performance gaps and societal implications, which were previously underexplored.

## Key findings

- VLMs perform poorly on street-level images.
- High accuracy (61%) on social media-like images.
- Raises privacy concerns due to model capabilities.

## Abstract

Geo-localization is the task of identifying the location of an image using visual cues alone. It has beneficial applications, such as improving disaster response, enhancing navigation, and geography education. Recently, Vision-Language Models (VLMs) are increasingly demonstrating capabilities as accurate image geo-locators. This brings significant privacy risks, including those related to stalking and surveillance, considering the widespread uses of AI models and sharing of photos on social media. The precision of these models is likely to improve in the future. Despite these risks, there is little work on systematically evaluating the geolocation precision of Generative VLMs, their limits and potential for unintended inferences. To bridge this gap, we conduct a comprehensive assessment of the geolocation capabilities of 25 state-of-the-art VLMs on four benchmark image datasets captured in diverse environments. Our results offer insight into the internal reasoning of VLMs and highlight their strengths, limitations, and potential societal risks. Our findings indicate that current VLMs perform poorly on generic street-level images yet achieve notably high accuracy (61\%) on images resembling social media content, raising significant and urgent privacy concerns.