ViSAGe: A Global-Scale Analysis of Visual Stereotypes in Text-to-Image Generation

TL;DR

This paper introduces ViSAGe, a comprehensive dataset for evaluating global stereotypes in Text-to-Image models, revealing significant biases and stereotypical depictions across 135 nationalities.

Contribution

The paper presents ViSAGe, a novel dataset that enables detailed analysis of nationality-based stereotypes in T2I models, covering a wide range of identities and stereotypes.

Findings

Stereotypical attributes are thrice as likely in generated images of corresponding identities.

Offensive stereotypes are more prevalent for identities from Africa, South America, and Southeast Asia.

Default representations tend to be stereotypical, especially for groups from the Global South.

Abstract

Recent studies have shown that Text-to-Image (T2I) model generations can reflect social stereotypes present in the real world. However, existing approaches for evaluating stereotypes have a noticeable lack of coverage of global identity groups and their associated stereotypes. To address this gap, we introduce the ViSAGe (Visual Stereotypes Around the Globe) dataset to enable the evaluation of known nationality-based stereotypes in T2I models, across 135 nationalities. We enrich an existing textual stereotype resource by distinguishing between stereotypical associations that are more likely to have visual depictions, such as `sombrero', from those that are less visually concrete, such as 'attractive'. We demonstrate ViSAGe's utility through a multi-faceted evaluation of T2I generations. First, we show that stereotypical attributes in ViSAGe are thrice as likely to be present in generated images of corresponding identities as compared to other attributes, and that the offensiveness of these depictions is especially higher for identities from Africa, South America, and South East Asia. Second, we assess the stereotypical pull of visual depictions of identity groups, which reveals how the 'default' representations of all identity groups in ViSAGe have a pull towards stereotypical depictions, and that this pull is even more prominent for identity groups from the Global South. CONTENT WARNING: Some examples contain offensive stereotypes.