Ambient Diffusion Omni: Training Good Models with Bad Data

TL;DR

This paper introduces Ambient Diffusion Omni, a framework that leverages low-quality and out-of-distribution images to enhance diffusion models, achieving state-of-the-art results by extracting valuable signals from all available data.

Contribution

The paper presents a novel training framework that utilizes low-quality images for diffusion models, with theoretical analysis and practical validation showing improved image quality and diversity.

Findings

Achieved state-of-the-art ImageNet FID scores.

Successfully trained models on synthetically corrupted images.

Significant improvements in image quality and diversity.

Abstract

We show how to use low-quality, synthetic, and out-of-distribution images to improve the quality of a diffusion model. Typically, diffusion models are trained on curated datasets that emerge from highly filtered data pools from the Web and other sources. We show that there is immense value in the lower-quality images that are often discarded. We present Ambient Diffusion Omni, a simple, principled framework to train diffusion models that can extract signal from all available images during training. Our framework exploits two properties of natural images -- spectral power law decay and locality. We first validate our framework by successfully training diffusion models with images synthetically corrupted by Gaussian blur, JPEG compression, and motion blur. We then use our framework to achieve state-of-the-art ImageNet FID, and we show significant improvements in both image quality and diversity for text-to-image generative modeling. The core insight is that noise dampens the initial skew between the desired high-quality distribution and the mixed distribution we actually observe. We provide rigorous theoretical justification for our approach by analyzing the trade-off between learning from biased data versus limited unbiased data across diffusion times.