Image-Perfect Imperfections: Safety, Bias, and Authenticity in the Shadow of Text-To-Image Model Evolution

TL;DR

This paper investigates how updates to Stable Diffusion affect safety, bias, and authenticity, revealing improvements in safety but increased bias and challenges in fake image detection, emphasizing ongoing mitigation needs.

Contribution

It provides a comprehensive analysis of the evolving safety, bias, and authenticity issues in Stable Diffusion, highlighting new challenges and solutions for fake image detection.

Findings

Updates reduce unsafe image generation

Bias in gender and race persists or shifts with updates

Fake image detectors need fine-tuning for new SD versions

Abstract

Text-to-image models, such as Stable Diffusion (SD), undergo iterative updates to improve image quality and address concerns such as safety. Improvements in image quality are straightforward to assess. However, how model updates resolve existing concerns and whether they raise new questions remain unexplored. This study takes an initial step in investigating the evolution of text-to-image models from the perspectives of safety, bias, and authenticity. Our findings, centered on Stable Diffusion, indicate that model updates paint a mixed picture. While updates progressively reduce the generation of unsafe images, the bias issue, particularly in gender, intensifies. We also find that negative stereotypes either persist within the same Non-White race group or shift towards other Non-White race groups through SD updates, yet with minimal association of these traits with the White race group. Additionally, our evaluation reveals a new concern stemming from SD updates: State-of-the-art fake image detectors, initially trained for earlier SD versions, struggle to identify fake images generated by updated versions. We show that fine-tuning these detectors on fake images generated by updated versions achieves at least 96.6\% accuracy across various SD versions, addressing this issue. Our insights highlight the importance of continued efforts to mitigate biases and vulnerabilities in evolving text-to-image models.