Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South

TL;DR

This paper introduces PLACES, a localized dataset of over 26,000 T2I model failures from the Global South, highlighting cultural-specific vulnerabilities and advocating for participatory safety frameworks.

Contribution

It presents a novel, community-centered approach to T2I safety, emphasizing localization, participation, and the creation of a culturally diverse failure dataset in the Global South.

Findings

Identification of unique adversarial patterns influenced by local culture and language

Discovery of structural gaps in existing safety frameworks regarding cultural norms

Demonstration of diverse socio-cultural and linguistic attributes in T2I failures

Abstract

Despite the global deployment of text-to-image (T2I) models, their safety frameworks are largely calibrated to a Western-centric default, creating significant vulnerabilities for the rest of the world. To embrace cultural pluralism and bring historically under-represented perspectives in T2I safety, we conduct localised community-centered red teaming studies in the Global South. Our two-fold approach prioritizes localization and participation, by focusing on secondary urban centers in these regions, and conducting community engagement and training workshops to contextualize local norms. As a result, we present PLACES, a dataset comprising over 26,000 examples of T2I model failures collected in partnership with universities in Ghana, Nigeria, and two regions of India (Karnataka and Punjab). Analysis of prompts collected reveals a wide-ranging diversity in socio-cultural and linguistic attributes, when compared to existing geography-agnostic crowdsourced red-teaming data. We observe unique adversarial patterns enabled by local cultural and linguistic nuances, and distinct clusters within region around specific themes, such as religion in India. Moreover, we uncover structural contextual gaps in existing safety frameworks by identifying novel harms showing normative dissonance (e.g., violating religious norms, ignoring local customs, and ominous symbolism). This work argues that expanding T2I safety requires moving beyond mere scale to incorporate deeply localised, participatory methodologies for data collection and contextualization. Content warning: This paper includes examples containing potentially harmful or offensive content.