Unbiased Image Synthesis via Manifold Guidance in Diffusion Models

TL;DR

This paper introduces a novel unsupervised sampling method called Manifold Guidance Sampling that reduces bias in diffusion model-generated images by leveraging data manifold structure, without retraining models.

Contribution

It presents the first unsupervised approach to mitigate bias in diffusion models through manifold guidance, improving diversity and fairness in generated images.

Findings

Reduces gender bias in CelebA dataset from 148% to more balanced levels.

Enhances image diversity and quality without retraining existing diffusion models.

Effectively disperses biased data clustering during sampling.

Abstract

Diffusion Models are a potent class of generative models capable of producing high-quality images. However, they often inadvertently favor certain data attributes, undermining the diversity of generated images. This issue is starkly apparent in skewed datasets like CelebA, where the initial dataset disproportionately favors females over males by 57.9%, this bias amplified in generated data where female representation outstrips males by 148%. In response, we propose a plug-and-play method named Manifold Guidance Sampling, which is also the first unsupervised method to mitigate bias issue in DDPMs. Leveraging the inherent structure of the data manifold, this method steers the sampling process towards a more uniform distribution, effectively dispersing the clustering of biased data. Without the need for modifying the existing model or additional training, it significantly mitigates data bias and enhances the quality and unbiasedness of the generated images.