GeoDiv: Framework For Measuring Geographical Diversity In Text-To-Image Models
Abhipsa Basu, Mohana Singh, Shashank Agnihotri, Margret Keuper, R. Venkatesh Babu
TL;DR
GeoDiv is a novel framework that assesses geographical diversity in text-to-image models using large language and vision-language models, revealing biases and lack of diversity in generated images across different regions.
Contribution
Introduces GeoDiv, the first systematic and interpretable framework for measuring geographical diversity and biases in text-to-image models.
Findings
Models often produce stereotyped and impoverished depictions of certain countries.
GeoDiv uncovers biases related to socio-economic conditions in generated images.
The framework highlights the need for more geographically nuanced generative models.
Abstract
Text-to-image (T2I) models are rapidly gaining popularity, yet their outputs often lack geographical diversity, reinforce stereotypes, and misrepresent regions. Given their broad reach, it is critical to rigorously evaluate how these models portray the world. Existing diversity metrics either rely on curated datasets or focus on surface-level visual similarity, limiting interpretability. We introduce GeoDiv, a framework leveraging large language and vision-language models to assess geographical diversity along two complementary axes: the Socio-Economic Visual Index (SEVI), capturing economic and condition-related cues, and the Visual Diversity Index (VDI), measuring variation in primary entities and backgrounds. Applied to images generated by models such as Stable Diffusion and FLUX.1-dev across entities and countries, GeoDiv reveals a consistent lack of diversity and…
Peer Reviews
Decision·ICLR 2026 Poster
- Auditing geo-diversity of generative models is important, but has not received sufficient attention, apart from a few research publications and media articles. - Analysis was performed on a large-scale set of generated images. The dataset involving 160k synthetic images across 16 countries × 10 entities, prompts, QA sets, and annotations could be valuable to the community. - Human validation with reasonable Spearman correlation (about 0.7) for the evaluation.
- The paper mentions bias and stereotypes interchangeably. However, these two concepts are different from each other as noted by OASIS. And, in the context of generative models, we are largely interested in stereotypes. Furthermore, measuring stereotypes depends on the baseline diversity of concepts in the real world, and deviations from this baseline. However, the current paper does not take this baseline into account, so it is unclear what exactly is being measured, and if the conclusions are
- The paper addresses an important issue of geographical diversity and socio-economic bias in text-to-image models, advancing beyond traditional demographic or visual diversity metrics toward a more region-aware fairness perspective. - The study builds a large-scale benchmark comprising 160,000 generated images across 16 countries, 10 object categories, and 4 text-to-image models, enabling a comprehensive analysis of model-, country-, and attribute-level biases, further validated through strong
- Novelty: - The framework’s core methodology lacks novelty. The idea heavily relies on prompt-based probing and LLM/VLM-assisted scoring, which have been explored in prior works such as [1] and [2], making the contribution more of an application to a new bias dimension rather than a methodological advance. - The proposed indices (SEVI and VDI) add interpretability but do not introduce fundamentally new techniques beyond existing entropy- or attribute-based bias quantification methods. Pr
1. The paper addresses the important and novel problem of quantifying geographical bias in T2I models by decoupling diversity into interpretable socio-economic and visual axes. 2. The proposed GeoDiv framework is rigorously validated against human judgments, building trust in its reliability. 3. The framework produces strong empirical results that highlight specific biases and demonstrate its advantages over existing diversity metrics. 4. The paper is clearly written, well-organized, and effecti
The paper's main weakness lies in its narrative structure, which could more clearly delineate the problem from the proposed solution. While the paper successfully demonstrates (1) that T2I models exhibit geographical bias and (2) that the GeoDiv framework is a valid VLM-based method to measure this, the presentation intertwines these two major points. A potentially stronger narrative might be: 1. First, establish the core problem: Demonstrate unequivocally that significant geographical bias exi
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsComputational and Text Analysis Methods · Language and cultural evolution · ICT in Developing Communities