Emergence of Distortions in High-Dimensional Guided Diffusion Models

TL;DR

This paper analyzes how classifier-free guidance causes distortions in high-dimensional diffusion models, revealing their dependence on data complexity and proposing improved guidance schedules.

Contribution

It provides a theoretical analysis of distortions in CFG, characterizes their emergence in high dimensions, and introduces a new guidance schedule to mitigate these issues.

Findings

Distortions emerge when classes scale exponentially with data dimension.

Distortions vanish in sub-exponential class regimes due to a phase transition.

A new guidance schedule with negative guidance improves sample diversity.

Abstract

Classifier-free guidance (CFG) is the de facto standard for conditional sampling in diffusion models, yet it often reduces sample diversity. Using tools from statistical physics, we analyze the emergence of generative distortions induced by CFG, namely the mismatch between the CFG sampling distribution and the true conditional distribution. We study this phenomenon in analytically tractable settings with exact score functions, characterizing its dependence on data dimensionality and the number of classes. For high-dimensional Gaussian mixtures, we use dynamic mean-field theory to show that distortions arise when the number of classes scales exponentially with the data dimension, whereas they vanish in the sub-exponential regime due to a dynamical phase transition. We further prove that, in the infinite-class limit, distortions remain unavoidable regardless of dimensionality because of the increasing density of classes. Finally, we show that standard CFG schedules cannot prevent variance shrinkage, and we propose a theoretically grounded guidance schedule incorporating a negative-guidance window that improves both class separability and sample diversity in real-world latent diffusion models.