How many measurements are enough? Bayesian recovery in inverse problems with general distributions

TL;DR

This paper provides a comprehensive analysis of the sample complexity required for Bayesian recovery in inverse problems, highlighting the roles of prior complexity, operator concentration, and noise, with special focus on deep neural network priors.

Contribution

It introduces a non-asymptotic bound for Bayesian inverse problems that accounts for general distributions and demonstrates the efficiency of DNN-based priors with logarithmic scaling in latent dimension.

Findings

Sample complexity depends on the approximate covering number of the prior.

DNN-based priors have sample complexity scaling log-linearly with latent dimension.

Coherence of the measurement matrix influences Bayesian recovery guarantees.

Abstract

We study the sample complexity of Bayesian recovery for solving inverse problems with general prior, forward operator and noise distributions. We consider posterior sampling according to an approximate prior $\mathcal{P}$, and establish sufficient conditions for stable and accurate recovery with high probability. Our main result is a non-asymptotic bound that shows that the sample complexity depends on (i) the intrinsic complexity of $\mathcal{P}$, quantified by its so-called approximate covering number, and (ii) concentration bounds for the forward operator and noise distributions. As a key application, we specialize to generative priors, where $\mathcal{P}$ is the pushforward of a latent distribution via a Deep Neural Network (DNN). We show that the sample complexity scales log-linearly with the latent dimension $k$, thus establishing the efficacy of DNN-based priors. Generalizing existing results on deterministic (i.e., non-Bayesian) recovery for the important problem of random sampling with an orthogonal matrix $U$, we show how the sample complexity is determined by the coherence of $U$ with respect to the support of $\mathcal{P}$. Hence, we establish that coherence plays a fundamental role in Bayesian recovery as well. Overall, our framework unifies and extends prior work, providing rigorous guarantees for the sample complexity of solving Bayesian inverse problems with arbitrary distributions.