Angle Domain Guidance: Latent Diffusion Requires Rotation Rather Than Extrapolation

TL;DR

This paper introduces Angle Domain Guidance (ADG), a novel method that improves text-to-image diffusion models by reducing color distortions caused by high guidance weights, while maintaining strong text-image alignment.

Contribution

The paper provides a theoretical analysis of norm amplification in classifier-free guidance and proposes ADG to optimize angular alignment, reducing distortions in generated images.

Findings

ADG reduces color distortions in high guidance weight images.

ADG maintains or improves text-image alignment quality.

Experimental results show ADG outperforms existing guidance methods.

Abstract

Classifier-free guidance (CFG) has emerged as a pivotal advancement in text-to-image latent diffusion models, establishing itself as a cornerstone technique for achieving high-quality image synthesis. However, under high guidance weights, where text-image alignment is significantly enhanced, CFG also leads to pronounced color distortions in the generated images. We identify that these distortions stem from the amplification of sample norms in the latent space. We present a theoretical framework that elucidates the mechanisms of norm amplification and anomalous diffusion phenomena induced by classifier-free guidance. Leveraging our theoretical insights and the latent space structure, we propose an Angle Domain Guidance (ADG) algorithm. ADG constrains magnitude variations while optimizing angular alignment, thereby mitigating color distortions while preserving the enhanced text-image alignment achieved at higher guidance weights. Experimental results demonstrate that ADG significantly outperforms existing methods, generating images that not only maintain superior text alignment but also exhibit improved color fidelity and better alignment with human perceptual preferences.