Frequency Regulation for Exposure Bias Mitigation in Diffusion Models

TL;DR

This paper introduces a training-free, frequency-based regulation method to mitigate exposure bias in diffusion models, improving generative quality with minimal computational overhead.

Contribution

It presents a novel, plug-and-play frequency regulation mechanism using wavelet transforms to address exposure bias in diffusion models.

Findings

Significant improvement in generative quality across various diffusion models.

The method is training-free and computationally efficient.

Provides a rigorous mathematical understanding of exposure bias.

Abstract

Diffusion models exhibit impressive generative capabilities but are significantly impacted by exposure bias. In this paper, we make a key observation: the energy of predicted noisy samples in the reverse process continuously declines compared to perturbed samples in the forward process. Building on this, we identify two important findings: 1) The reduction in energy follows distinct patterns in the low-frequency and high-frequency subbands; 2) The subband energy of reverse-process reconstructed samples is consistently lower than that of forward-process ones, and both are lower than the original data samples. Based on the first finding, we introduce a dynamic frequency regulation mechanism utilizing wavelet transforms, which separately adjusts the low- and high-frequency subbands. Leveraging the second insight, we derive the rigorous mathematical form of exposure bias. It is worth noting that, our method is training-free and plug-and-play, significantly improving the generative quality of various diffusion models and frameworks with negligible computational cost. The source code is available at https://github.com/kunzhan/wpp.