Conditional Diffusion Models are Minimax-Optimal and Manifold-Adaptive for Conditional Distribution Estimation

TL;DR

This paper analyzes conditional diffusion models within a statistical framework, proving their minimax-optimal convergence rates and manifold-adaptive capabilities for conditional distribution estimation.

Contribution

It provides a theoretical analysis of conditional diffusion models, establishing their optimality and adaptability to low-dimensional manifold structures in data and covariates.

Findings

Conditional diffusion models achieve minimax-optimal convergence rates.

Models can adapt to low-dimensional manifold structures.

Estimation error depends only on intrinsic dimensions.

Abstract

We consider a class of conditional forward-backward diffusion models for conditional generative modeling, that is, generating new data given a covariate (or control variable). To formally study the theoretical properties of these conditional generative models, we adopt a statistical framework of distribution regression to characterize the large sample properties of the conditional distribution estimators induced by these conditional forward-backward diffusion models. Here, the conditional distribution of data is assumed to smoothly change over the covariate. In particular, our derived convergence rate is minimax-optimal under the total variation metric within the regimes covered by the existing literature. Additionally, we extend our theory by allowing both the data and the covariate variable to potentially admit a low-dimensional manifold structure. In this scenario, we demonstrate that the conditional forward-backward diffusion model can adapt to both manifold structures, meaning that the derived estimation error bound (under the Wasserstein metric) depends only on the intrinsic dimensionalities of the data and the covariate.