Towards a unified framework for guided diffusion models

TL;DR

This paper develops a unified theoretical framework for guided diffusion models, explaining how guidance improves data generation and providing new insights into classifier-free guidance and reward-based sampling.

Contribution

It introduces a unified algorithmic and theoretical framework for guided diffusion, including reward guidance and classifier-free guidance, with rigorous analysis and practical algorithms.

Findings

Classifier-free guidance decreases the reciprocal of classifier probability.

The framework quantifies reward improvements over unguided diffusion.

A new reward-guided sampler that is easy to train and efficient.

Abstract

Guided or controlled data generation with diffusion models\blfootnote{Partial preliminary results of this work appeared in International Conference on Machine Learning 2025 \citep{li2025provable}.} has become a cornerstone of modern generative modeling. Despite substantial advances in diffusion model theory, the theoretical understanding of guided diffusion samplers remains severely limited. We make progress by developing a unified algorithmic and theoretical framework that accommodates both diffusion guidance and reward-guided diffusion. Aimed at fine-tuning diffusion models to improve certain rewards, we propose injecting a reward guidance term -- constructed from the difference between the original and reward-reweighted scores -- into the backward diffusion process, and rigorously quantify the resulting reward improvement over the unguided counterpart. As a key application, our framework shows that classifier-free guidance (CFG) decreases the expected reciprocal of the classifier probability, providing the first theoretical characterization of the specific performance metric that CFG improves for general target distributions. When applied to reward-guided diffusion, our framework yields a new sampler that is easy-to-train and requires no full diffusion trajectories during training. Numerical experiments further corroborate our theoretical findings.