Improved Denoising Diffusion Probabilistic Models

TL;DR

This paper introduces simple modifications to diffusion probabilistic models that improve their likelihood performance, reduce sampling steps significantly, and demonstrate scalable high-quality sample generation, enhancing their practicality and comparison with GANs.

Contribution

The authors propose modifications to DDPMs that improve likelihoods, enable fewer sampling steps, and provide comprehensive evaluation metrics, advancing the state of diffusion models.

Findings

Modified DDPMs achieve competitive log-likelihoods.

Sampling with fewer forward passes is possible with negligible quality loss.

Model performance scales smoothly with capacity and compute.

Abstract

Denoising diffusion probabilistic models (DDPM) are a class of generative models which have recently been shown to produce excellent samples. We show that with a few simple modifications, DDPMs can also achieve competitive log-likelihoods while maintaining high sample quality. Additionally, we find that learning variances of the reverse diffusion process allows sampling with an order of magnitude fewer forward passes with a negligible difference in sample quality, which is important for the practical deployment of these models. We additionally use precision and recall to compare how well DDPMs and GANs cover the target distribution. Finally, we show that the sample quality and likelihood of these models scale smoothly with model capacity and training compute, making them easily scalable. We release our code at https://github.com/openai/improved-diffusion