On Error Propagation of Diffusion Models

TL;DR

This paper develops a theoretical framework to analyze error propagation in diffusion models, demonstrating how cumulative error impacts generation quality and proposing a regularization method to mitigate this issue, leading to improved performance.

Contribution

It introduces a novel theoretical framework for error propagation in diffusion models and proposes an effective regularization technique to enhance their performance.

Findings

Regularization reduces error propagation in diffusion models

Proposed method improves image generation quality

Outperforms previous baseline methods

Abstract

Although diffusion models (DMs) have shown promising performances in a number of tasks (e.g., speech synthesis and image generation), they might suffer from error propagation because of their sequential structure. However, this is not certain because some sequential models, such as Conditional Random Field (CRF), are free from this problem. To address this issue, we develop a theoretical framework to mathematically formulate error propagation in the architecture of DMs, The framework contains three elements, including modular error, cumulative error, and propagation equation. The modular and cumulative errors are related by the equation, which interprets that DMs are indeed affected by error propagation. Our theoretical study also suggests that the cumulative error is closely related to the generation quality of DMs. Based on this finding, we apply the cumulative error as a regularization term to reduce error propagation. Because the term is computationally intractable, we derive its upper bound and design a bootstrap algorithm to efficiently estimate the bound for optimization. We have conducted extensive experiments on multiple image datasets, showing that our proposed regularization reduces error propagation, significantly improves vanilla DMs, and outperforms previous baselines.