Non-Normal Diffusion Models

TL;DR

This paper explores the impact of the diffusion step distribution in diffusion models, revealing that relaxing the normality assumption enhances model flexibility and affects generated sample quality.

Contribution

It introduces a generalized diffusion framework by removing the normality assumption on step sizes, expanding the design space and training options for diffusion models.

Findings

Different step distributions produce qualitatively different samples

Generalized models improve density estimation and image generation

Invariance to step distribution as step size approaches zero

Abstract

Diffusion models generate samples by incrementally reversing a process that turns data into noise. We show that when the step size goes to zero, the reversed process is invariant to the distribution of these increments. This reveals a previously unconsidered parameter in the design of diffusion models: the distribution of the diffusion step $\Delta x_k := x_{k} - x_{k + 1}$. This parameter is implicitly set by default to be normally distributed in most diffusion models. By lifting this assumption, we generalize the framework for designing diffusion models and establish an expanded class of diffusion processes with greater flexibility in the choice of loss function used during training. We demonstrate the effectiveness of these models on density estimation and generative modeling tasks on standard image datasets, and show that different choices of the distribution of $\Delta x_k$ result in qualitatively different generated samples.