Stable Diffusion Exposed: Gender Bias from Prompt to Image

TL;DR

This paper investigates gender bias in Stable Diffusion image generation, analyzing how gender indicators influence representations, object depiction, and layout, and offers insights and recommendations to mitigate bias.

Contribution

Introduces an evaluation protocol for analyzing gender bias at each step of Stable Diffusion image generation, highlighting biases related to object depiction and layout.

Findings

Gender indicators affect object representation and layout.

Neutral prompts often produce images more aligned with masculine stereotypes.

Bias originates from prompt-image dependencies and representation disparities.

Abstract

Several studies have raised awareness about social biases in image generative models, demonstrating their predisposition towards stereotypes and imbalances. This paper contributes to this growing body of research by introducing an evaluation protocol that analyzes the impact of gender indicators at every step of the generation process on Stable Diffusion images. Leveraging insights from prior work, we explore how gender indicators not only affect gender presentation but also the representation of objects and layouts within the generated images. Our findings include the existence of differences in the depiction of objects, such as instruments tailored for specific genders, and shifts in overall layouts. We also reveal that neutral prompts tend to produce images more aligned with masculine prompts than their feminine counterparts. We further explore where bias originates through representational disparities and how it manifests in the images via prompt-image dependencies, and provide recommendations for developers and users to mitigate potential bias in image generation.