Adaptive Destruction Processes for Diffusion Samplers

TL;DR

This paper introduces a trainable, discrete-time diffusion process with adaptive destruction mechanisms, enhancing sampling efficiency and quality in generative models by decoupling generation and destruction variances.

Contribution

It proposes a flexible, trainable diffusion sampler that decouples generation and destruction variances, improving convergence and sample quality with fewer steps.

Findings

Faster convergence with limited steps.

Improved sampling quality on benchmarks.

Scalability demonstrated on GAN latent spaces.

Abstract

This paper explores the challenges and benefits of a trainable destruction process in diffusion samplers -- diffusion-based generative models trained to sample an unnormalised density without access to data samples. Contrary to the majority of work that views diffusion samplers as approximations to an underlying continuous-time model, we view diffusion models as discrete-time policies trained to produce samples in very few generation steps. We propose to trade some of the elegance of the underlying theory for flexibility in the definition of the generative and destruction policies. In particular, we decouple the generation and destruction variances, enabling both transition kernels to be learned as unconstrained Gaussian densities. We show that, when the number of steps is limited, training both generation and destruction processes results in faster convergence and improved sampling quality on various benchmarks. Through a robust ablation study, we investigate the design choices necessary to facilitate stable training. Finally, we show the scalability of our approach through experiments on GAN latent space sampling for conditional image generation.