Generative Uncertainty in Diffusion Models

TL;DR

This paper introduces a Bayesian framework to estimate uncertainty in diffusion models, enabling the detection of low-quality samples without human inspection, and improves the reliability of generative outputs.

Contribution

It presents a practical Bayesian inference method with a new semantic likelihood for large diffusion models, enhancing sample quality assessment post-hoc.

Findings

Effective identification of poor-quality samples

Outperforms existing uncertainty methods

Applicable to any pretrained diffusion or flow model

Abstract

Diffusion models have recently driven significant breakthroughs in generative modeling. While state-of-the-art models produce high-quality samples on average, individual samples can still be low quality. Detecting such samples without human inspection remains a challenging task. To address this, we propose a Bayesian framework for estimating generative uncertainty of synthetic samples. We outline how to make Bayesian inference practical for large, modern generative models and introduce a new semantic likelihood (evaluated in the latent space of a feature extractor) to address the challenges posed by high-dimensional sample spaces. Through our experiments, we demonstrate that the proposed generative uncertainty effectively identifies poor-quality samples and significantly outperforms existing uncertainty-based methods. Notably, our Bayesian framework can be applied post-hoc to any pretrained diffusion or flow matching model (via the Laplace approximation), and we propose simple yet effective techniques to minimize its computational overhead during sampling.