Non-asymptotic convergence bound of conditional diffusion models

TL;DR

This paper establishes non-asymptotic convergence bounds for conditional diffusion models, providing a theoretical foundation for their analysis and demonstrating their effectiveness in learning conditional distributions.

Contribution

It introduces a theoretical framework for conditional diffusion models, deriving convergence bounds and connecting them with the Fokker-Planck equation under Lipschitz assumptions.

Findings

Derived stochastic differential equations for CARD

Established upper error bounds using Wasserstein distance

Provided convergence bounds under additional distribution assumptions

Abstract

Learning and generating various types of data based on conditional diffusion models has been a research hotspot in recent years. Although conditional diffusion models have made considerable progress in improving acceleration algorithms and enhancing generation quality, the lack of non-asymptotic properties has hindered theoretical research. To address this gap, we focus on a conditional diffusion model within the domains of classification and regression (CARD), which aims to learn the original distribution with given input x (denoted as Y|X). It innovatively integrates a pre-trained model f_{\phi}(x) into the original diffusion model framework, allowing it to precisely capture the original conditional distribution given f (expressed as Y|f_{\phi}(x)). Remarkably, when f_{\phi}(x) performs satisfactorily, Y|f_{\phi}(x) closely approximates Y|X. Theoretically, we deduce the stochastic differential equations of CARD and establish its generalized form predicated on the Fokker-Planck equation, thereby erecting a firm theoretical foundation for analysis. Mainly under the Lipschitz assumptions, we utilize the second-order Wasserstein distance to demonstrate the upper error bound between the original and the generated conditional distributions. Additionally, by appending assumptions such as light-tailedness to the original distribution, we derive the convergence upper bound between the true value analogous to the score function and the corresponding network-estimated value.