Enhancing diffusion models with Gaussianization preprocessing

TL;DR

This paper introduces Gaussianization preprocessing to training data for diffusion models, aiming to improve early-stage reconstruction quality and sampling efficiency, especially in small networks.

Contribution

It proposes a novel Gaussianization preprocessing method that simplifies the learning task of diffusion models, enhancing early-stage generation quality and stability.

Findings

Improved early-stage reconstruction quality in diffusion models.

Enhanced sampling stability and efficiency.

Effective for small-scale network architectures.

Abstract

Diffusion models are a class of generative models that have demonstrated remarkable success in tasks such as image generation. However, one of the bottlenecks of these models is slow sampling due to the delay before the onset of trajectory bifurcation, at which point substantial reconstruction begins. This issue degrades generation quality, especially in the early stages. Our primary objective is to mitigate bifurcation-related issues by preprocessing the training data to enhance reconstruction quality, particularly for small-scale network architectures. Specifically, we propose applying Gaussianization preprocessing to the training data to make the target distribution more closely resemble an independent Gaussian distribution, which serves as the initial density of the reconstruction process. This preprocessing step simplifies the model's task of learning the target distribution, thereby improving generation quality even in the early stages of reconstruction with small networks. The proposed method is, in principle, applicable to a broad range of generative tasks, enabling more stable and efficient sampling processes.