Convergence of the Inexact Langevin Algorithm in KL Divergence with Application to Score-based Generative Models

TL;DR

This paper analyzes the convergence of the Inexact Langevin Algorithm in KL divergence for score-based generative models, establishing stable biased convergence guarantees under certain assumptions and exploring score estimator accuracy.

Contribution

It provides the first stable convergence guarantees for ILD and ILA in KL divergence under log-Sobolev and MGF error assumptions, and proposes a kernel density estimator for score accuracy.

Findings

Stable biased convergence in KL divergence under log-Sobolev inequality.

Bounded convergence in Rényi divergence under stronger score error assumptions.

Kernel density estimator can achieve MGF error assumption for sub-Gaussian targets.

Abstract

Motivated by the increasingly popular Score-based Generative Modeling (SGM), we study the Inexact Langevin Dynamics (ILD) and Inexact Langevin Algorithm (ILA) where a score function estimate is used in place of the exact score. We establish {\em stable} biased convergence guarantees in terms of the Kullback-Leibler (KL) divergence. To achieve these guarantees, we impose two key assumptions: 1) the target distribution satisfies the log-Sobolev inequality, and 2) the error of score estimator exhibits a sub-Gaussian tail, referred to as Moment Generating Function (MGF) error assumption. Under the stronger $L^\infty$ score error assumption, we obtain a stable convergence bound in R\'enyi divergence. We also generalize the proof technique to SGM, and derive a stable convergence bound in KL divergence. In addition, we explore the question of how to obtain a provably accurate score estimator. We demonstrate that a simple estimator based on kernel density estimation fulfills the MGF error assumption for sub-Gaussian target distributions, at the population level.