Non-asymptotic Analysis of Diffusion Annealed Langevin Monte Carlo for Generative Modelling

TL;DR

This paper provides non-asymptotic error bounds and theoretical guarantees for diffusion annealed Langevin Monte Carlo (DALMC), including heavy-tailed diffusion paths, for score-based generative models interpolating between simple and complex data distributions.

Contribution

It introduces non-asymptotic analysis of DALMC with Gaussian and Student's t diffusion paths, extending theoretical understanding to heavy-tailed data distributions.

Findings

Non-asymptotic error bounds for DALMC with Gaussian paths.

Extension of theoretical guarantees to heavy-tailed Student's t diffusion paths.

Broader perspective on score-based generative models beyond Ornstein-Uhlenbeck processes.

Abstract

We investigate the theoretical properties of general diffusion (interpolation) paths and their Langevin Monte Carlo implementation, referred to as diffusion annealed Langevin Monte Carlo (DALMC), under weak conditions on the data distribution. Specifically, we analyse and provide non-asymptotic error bounds for the annealed Langevin dynamics where the path of distributions is defined as Gaussian convolutions of the data distribution as in diffusion models. We then extend our results to recently proposed heavy-tailed (Student's t) diffusion paths, demonstrating their theoretical properties for heavy-tailed data distributions for the first time. Our analysis provides theoretical guarantees for a class of score-based generative models that interpolate between a simple distribution (Gaussian or Student's t) and the data distribution in finite time. This approach offers a broader perspective compared to standard score-based diffusion approaches, which are typically based on a forward Ornstein-Uhlenbeck (OU) noising process.