A Complete Recipe for Diffusion Generative Models

TL;DR

This paper provides a comprehensive framework for designing forward diffusion processes in score-based generative models, introduces Phase Space Langevin Diffusion (PSLD), and demonstrates its superior performance in image synthesis tasks.

Contribution

It offers a complete recipe for forward processes in SGMs, unifies existing models under this framework, and introduces PSLD with improved sample quality and efficiency.

Findings

PSLD achieves state-of-the-art FID scores on CIFAR-10.

The framework unifies various existing SGMs.

PSLD offers a better speed-quality trade-off.

Abstract

Score-based Generative Models (SGMs) have demonstrated exceptional synthesis outcomes across various tasks. However, the current design landscape of the forward diffusion process remains largely untapped and often relies on physical heuristics or simplifying assumptions. Utilizing insights from the development of scalable Bayesian posterior samplers, we present a complete recipe for formulating forward processes in SGMs, ensuring convergence to the desired target distribution. Our approach reveals that several existing SGMs can be seen as specific manifestations of our framework. Building upon this method, we introduce Phase Space Langevin Diffusion (PSLD), which relies on score-based modeling within an augmented space enriched by auxiliary variables akin to physical phase space. Empirical results exhibit the superior sample quality and improved speed-quality trade-off of PSLD compared to various competing approaches on established image synthesis benchmarks. Remarkably, PSLD achieves sample quality akin to state-of-the-art SGMs (FID: 2.10 for unconditional CIFAR-10 generation). Lastly, we demonstrate the applicability of PSLD in conditional synthesis using pre-trained score networks, offering an appealing alternative as an SGM backbone for future advancements. Code and model checkpoints can be accessed at \url{https://github.com/mandt-lab/PSLD}.