Mitigating Inappropriateness in Image Generation: Can there be Value in Reflecting the World's Ugliness?

TL;DR

This paper investigates the prevalence of inappropriate content in text-to-image models and explores mitigation strategies that leverage models' understanding of the world's ugliness to better align with human preferences.

Contribution

It demonstrates the extent of inappropriate degeneration in large-scale models and proposes mitigation methods that utilize models' representations of ugliness for improved alignment.

Findings

Models exhibit significant inappropriate content generation.

Mitigation strategies can effectively reduce inappropriate outputs.

Using models' perception of ugliness helps align outputs with human values.

Abstract

Text-conditioned image generation models have recently achieved astonishing results in image quality and text alignment and are consequently employed in a fast-growing number of applications. Since they are highly data-driven, relying on billion-sized datasets randomly scraped from the web, they also reproduce inappropriate human behavior. Specifically, we demonstrate inappropriate degeneration on a large-scale for various generative text-to-image models, thus motivating the need for monitoring and moderating them at deployment. To this end, we evaluate mitigation strategies at inference to suppress the generation of inappropriate content. Our findings show that we can use models' representations of the world's ugliness to align them with human preferences.