Diffusion annealed Langevin dynamics: a theoretical study

TL;DR

This paper provides a rigorous theoretical analysis of diffusion annealed Langevin dynamics, establishing existence, uniqueness, and inequalities that underpin its efficiency for generative modeling.

Contribution

It offers a formal construction of diffusion annealed Langevin dynamics and analyzes its efficiency through Poincaré and logarithmic Sobolev inequalities.

Findings

Proved existence and uniqueness of solutions for the diffusion process.

Established a Poincaré inequality for conditional distributions.

Showed that strengthening to a logarithmic Sobolev inequality enhances efficiency.

Abstract

In this work we study the diffusion annealed Langevin dynamics, a score-based diffusion process recently introduced in the theory of generative models and which is an alternative to the classical overdamped Langevin diffusion. Our goal is to provide a rigorous construction and to study the theoretical efficiency of these models for general base distribution as well as target distribution. As a matter of fact these diffusion processes are a particular case of Nelson processes i.e. diffusion processes with a given flow of time marginals. Providing existence and uniqueness of the solution to the annealed Langevin diffusion leads to proving a Poincar\'e inequality for the conditional distribution of $X$ knowing $X+Z=y$ uniformly in $y$, as recently observed by one of us and her coauthors. Part of this work is thus devoted to the study of such Poincar\'e inequalities. Additionally we show that strengthening the Poincar\'e inequality into a logarithmic Sobolev inequality improves the efficiency of the model.