BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inference

TL;DR

BayesDiff introduces a Bayesian inference-based pixel-wise uncertainty estimator for diffusion models, enabling better quality assessment and artifact correction in image generation tasks.

Contribution

It proposes a novel uncertainty iteration principle and uses Laplace approximation for efficient pixel-wise uncertainty estimation in diffusion models.

Findings

Effective in filtering low-fidelity images

Aids in artifact correction and generation augmentation

Demonstrated promising results in practical applications

Abstract

Diffusion models have impressive image generation capability, but low-quality generations still exist, and their identification remains challenging due to the lack of a proper sample-wise metric. To address this, we propose BayesDiff, a pixel-wise uncertainty estimator for generations from diffusion models based on Bayesian inference. In particular, we derive a novel uncertainty iteration principle to characterize the uncertainty dynamics in diffusion, and leverage the last-layer Laplace approximation for efficient Bayesian inference. The estimated pixel-wise uncertainty can not only be aggregated into a sample-wise metric to filter out low-fidelity images but also aids in augmenting successful generations and rectifying artifacts in failed generations in text-to-image tasks. Extensive experiments demonstrate the efficacy of BayesDiff and its promise for practical applications.