Cloud Diffusion Part 1: Theory and Motivation

TL;DR

This paper introduces the concept of Cloud Diffusion Models, which replace white noise with scale-invariant noise profiles in diffusion models, potentially enhancing image generation by capturing natural image statistics more effectively.

Contribution

It proposes a novel class of diffusion models utilizing scale-invariant noise profiles, offering theoretical motivation and potential advantages over traditional white noise diffusion models.

Findings

Theoretical framework for scale-invariant noise in diffusion models

Potential for faster inference and better detail in generated images

Future work to empirically compare Cloud Diffusion Models with classic models

Abstract

Diffusion models for image generation function by progressively adding noise to an image set and training a model to separate out the signal from the noise. The noise profile used by these models is white noise -- that is, noise based on independent normal distributions at each point whose mean and variance is independent of the scale. By contrast, most natural image sets exhibit a type of scale invariance in their low-order statistical properties characterized by a power-law scaling. Consequently, natural images are closer (in a quantifiable sense) to a different probability distribution that emphasizes large scale correlations and de-emphasizes small scale correlations. These scale invariant noise profiles can be incorporated into diffusion models in place of white noise to form what we will call a ``Cloud Diffusion Model". We argue that these models can lead to faster inference, improved high-frequency details, and greater controllability. In a follow-up paper, we will build and train a Cloud Diffusion Model that uses scale invariance at a fundamental level and compare it to classic, white noise diffusion models.