Provable Efficiency of Guidance in Diffusion Models for General Data Distribution

TL;DR

This paper provides a theoretical analysis of guidance in diffusion models, showing it can improve overall sample quality for general data distributions, beyond simple case studies.

Contribution

It offers the first theoretical framework for understanding guidance effects in diffusion models across general data distributions, extending beyond previous specific cases.

Findings

Guidance reduces the average reciprocal of classifier probability.

Guidance can improve overall sample quality in general distributions.

Theoretical results align with the practical motivation for guidance.

Abstract

Diffusion models have emerged as a powerful framework for generative modeling, with guidance techniques playing a crucial role in enhancing sample quality. Despite their empirical success, a comprehensive theoretical understanding of the guidance effect remains limited. Existing studies only focus on case studies, where the distribution conditioned on each class is either isotropic Gaussian or supported on a one-dimensional interval with some extra conditions. How to analyze the guidance effect beyond these case studies remains an open question. Towards closing this gap, we make an attempt to analyze diffusion guidance under general data distributions. Rather than demonstrating uniform sample quality improvement, which does not hold in some distributions, we prove that guidance can improve the whole sample quality, in the sense that the average reciprocal of the classifier probability decreases with the existence of guidance. This aligns with the motivation of introducing guidance.