Rethinking Oversaturation in Classifier-Free Guidance via Low Frequency

TL;DR

This paper presents LF-CFG, a novel method that reduces oversaturation in classifier-free guidance by adaptively down-weighting redundant low-frequency signals, improving image quality in diffusion models.

Contribution

It introduces a low-frequency perspective and an adaptive threshold-based approach to mitigate oversaturation in classifier-free guidance for diffusion models.

Findings

LF-CFG effectively reduces oversaturation and artifacts.

It improves image quality across multiple diffusion models.

The method is adaptable to various model versions.

Abstract

Classifier-free guidance (CFG) succeeds in condition diffusion models that use a guidance scale to balance the influence of conditional and unconditional terms. A high guidance scale is used to enhance the performance of the conditional term. However, the high guidance scale often results in oversaturation and unrealistic artifacts. In this paper, we introduce a new perspective based on low-frequency signals, identifying the accumulation of redundant information in these signals as the key factor behind oversaturation and unrealistic artifacts. Building on this insight, we propose low-frequency improved classifier-free guidance (LF-CFG) to mitigate these issues. Specifically, we introduce an adaptive threshold-based measurement to pinpoint the locations of redundant information. We determine a reasonable threshold by analyzing the change rate of low-frequency information between prior and current steps. We then apply a down-weight strategy to reduce the impact of redundant information in the low-frequency signals. Experimental results demonstrate that LF-CFG effectively alleviates oversaturation and unrealistic artifacts across various diffusion models, including Stable Diffusion-XL, Stable Diffusion 2.1, 3.0, 3.5, and SiT-XL.