Optimizing Diffusion Priors in Image Reconstruction from a Single Observation

TL;DR

This paper introduces a method to fine-tune diffusion priors from a single observation by combining multiple priors through a product-of-experts approach and optimizing Bayesian evidence, enhancing image reconstruction quality.

Contribution

The authors propose a novel technique for adapting diffusion priors using only one observation, enabling more flexible and trustworthy posterior sampling in inverse problems.

Findings

Maximized Bayesian evidence by adjusting prior exponents.

Improved image reconstruction in real-world inverse problems.

Extended priors beyond those trained on single datasets.

Abstract

While diffusion priors generate high-quality posterior samples across many inverse problems, they are often trained on limited training sets or purely simulated data, thus inheriting the errors and biases of these underlying sources. Current approaches to finetuning diffusion models rely on a large number of observations with varying forward operators, which can be difficult to collect for many applications, and thus lead to overfitting when the measurement set is small. We propose a method for tuning a prior from only a single observation by combining existing diffusion priors into a single product-of-experts prior and identifying the exponents that maximize the Bayesian evidence. We validate our method on real-world inverse problems, including black hole imaging, where the true prior is unknown a priori, and image deblurring with text-conditioned priors. We find that the evidence is often maximized by priors that extend beyond those trained on a single dataset. By generalizing the prior through exponent weighting, our approach enables posterior sampling from both tempered and combined diffusion models, yielding more flexible priors that improve the trustworthiness of the resulting posterior image distribution.