Weak Diffusion Priors Can Still Achieve Strong Inverse-Problem Performance

TL;DR

This paper investigates the surprising effectiveness of weak diffusion priors in inverse problems, showing they perform well with highly informative measurements and providing theoretical conditions for their reliability.

Contribution

It offers a theoretical framework explaining when and why weak diffusion priors can achieve strong inverse-problem performance, supported by extensive experiments.

Findings

Weak priors succeed with highly informative measurements

Identifies regimes where weak priors fail

Provides Bayesian theory for posterior concentration

Abstract

Can a diffusion model trained on bedrooms recover human faces? Diffusion models are widely used as priors for inverse problems, but standard approaches usually assume a high-fidelity model trained on data that closely match the unknown signal. In practice, one often must use a mismatched or low-fidelity diffusion prior. Surprisingly, these weak priors often perform nearly as well as full-strength, in-domain baselines. We study when and why inverse solvers are robust to weak diffusion priors. Through extensive experiments, we find that weak priors succeed when measurements are highly informative (e.g., many observed pixels), and we identify regimes where they fail. Our theory, based on Bayesian consistency, gives conditions under which high-dimensional measurements make the posterior concentrate near the true signal. These results provide a principled justification on when weak diffusion priors can be used reliably.