Sparsity-promoting and edge-preserving maximum a posteriori estimators in non-parametric Bayesian inverse problems

TL;DR

This paper establishes that in non-parametric Bayesian inverse problems with Besov space priors, MAP estimates can be characterized as minimizers of a generalized functional, even with non-Gaussian priors, and demonstrates their consistency.

Contribution

It proves that MAP estimates in non-parametric Bayesian inverse problems with Besov priors are characterized by a generalized Onsager--Machlup functional, extending previous results to non-Gaussian priors.

Findings

MAP estimates are characterized by a generalized Onsager--Machlup functional.

Weak and strong MAP estimates coincide in this setting.

Weak consistency of MAP estimators is proven in the high data limit.

Abstract

We consider the inverse problem of recovering an unknown functional parameter $u$ in a separable Banach space, from a noisy observation $y$ of its image through a known possibly non-linear ill-posed map ${\mathcal G}$. The data $y$ is finite-dimensional and the noise is Gaussian. We adopt a Bayesian approach to the problem and consider Besov space priors (see Lassas et al. 2009), which are well-known for their edge-preserving and sparsity-promoting properties and have recently attracted wide attention especially in the medical imaging community. Our key result is to show that in this non-parametric setup the maximum a posteriori (MAP) estimates are characterized by the minimizers of a generalized Onsager--Machlup functional of the posterior. This is done independently for the so-called weak and strong MAP estimates, which as we show coincide in our context. In addition, we prove a form of weak consistency for the MAP estimators in the infinitely informative data limit. Our results are remarkable for two reasons: first, the prior distribution is non-Gaussian and does not meet the smoothness conditions required in previous research on non-parametric MAP estimates. Second, the result analytically justifies existing uses of the MAP estimate in finite but high dimensional discretizations of Bayesian inverse problems with the considered Besov priors.

Equations

$$y=\mathcal{G}(u)+\xi ,$$

$$\mu^{y}(du|y)\propto\exp\big{(}-\Phi(u;y)\big{)}\mu_{0}(du),$$

$$\Phi(u;y)=\frac{1}{2}\left\|\Sigma^{-\frac{1}{2}}\big{(}y-\mathcal{G}(u)\big{)}\right\|^{2}_{Y}.$$

$$\pi (u)\propto \exp (-\alpha {\Vert u\Vert }_{BV}),$$

$$\pi (u)\propto \exp (-{\Vert u\Vert }_{B_p^s}^p),$$

$$\pi (u)\propto \exp (-W(u)),$$

$${\pi }^y(u)\propto \exp (-I(u;y)),$$

$$I(u;y)=\Phi (u;y)+W(u).$$

$$\underset{ϵ\to 0}{\lim }\frac{\mu (B_{ϵ}(z_2))}{\mu (B_{ϵ}(z_1))}=\exp (I(z_1)-I(z_2)),$$

$$\lim_{\epsilon\to 0}\frac{\mu(B_{\epsilon}(u-h))}{\mu(B_{\epsilon}(u))}\quad\mbox{is continuous in $u\in X$ for any fixed $h\in E$},$$

$$I(u;y)=\Phi (u;y)+{\Vert u\Vert }_{B_1^s}.$$

$$Q(\mu )=\{h|\mu \text{is quasi-invariant along}h\},$$

$$\underset{ϵ\to 0}{\lim }\frac{\mu (B_{ϵ}(\hat{u}))}{M^{ϵ}}=1.$$

$$\underset{ϵ\to 0}{\lim }\frac{\mu (B_{ϵ}(\hat{u}-h))}{\mu (B_{ϵ}(\hat{u}))}\le 1,$$

$$\underset{ϵ\to 0}{\lim }\frac{{\mu }_h(A^{ϵ}+u)}{\mu (A^{ϵ}+u)}={\tilde{R}}_h^{\mu }(u)$$

$$\underset{ϵ\to 0}{\lim }\frac{\mu (B_{ϵ}(\hat{u}))}{M_{\delta }^{ϵ}}=1.$$

$$\underset{ϵ\to 0}{\lim }\frac{\mu (B_{ϵ}(\hat{u}-h))}{\mu (B_{ϵ}(\hat{u}))}\le 1$$

$$\mu (\lambda A+(1-\lambda )B)\ge \mu (A{)}^{\lambda }\mu (B{)}^{1-\lambda },$$

$$\mu (\lambda A+(1-\lambda )B)\ge \min \{\mu (A),\mu (B)\},$$

$$\mu (A+x)\le \mu (A),\text{mbox}foranyx\in X.$$

$$f((1-\lambda )x+\lambda y)\ge \min \{f(x),f(y)\}.$$

$$f(x)=\sum _{\ell =1}^{\infty }{c}_{\ell }{\psi }_{\ell }(x)$$

$${\Vert f\Vert }_{B_p^s({\mathbb{T}}^d)}={\left(\sum _{\ell =1}^{\infty }{\ell }^{p(\frac{s}{d}+\frac{1}{2})-1}|c_{\ell }{|}^p\right)}^{\frac{1}{p}},$$

$${\pi }_X(x)=\frac{1}{2}\exp (-|x|).$$

$$U(x)=\sum _{\ell =1}^{\infty }{\ell }^{-(\frac{s}{d}-\frac{1}{2})}{X}_{\ell }{\psi }_{\ell }(x),x\in {\mathbb{T}}^d.$$

$$\frac{d\lambda_{h}}{d\lambda}(u)=\lim_{N\to\infty}\exp\sum_{\ell=1}^{N}\big{(}-\alpha_{\ell}|h_{\ell}-u_{\ell}|+\alpha_{\ell}|u_{\ell}|\big{)}$$

$$\underset{ϵ\to 0}{\lim }\frac{{\lambda }_h(B_{ϵ}(u))}{\lambda (B_{ϵ}(u))}=\exp \sum _{\ell =1}^{\infty }(-{\alpha }_{\ell }|h_{\ell }-u_{\ell }|+{\alpha }_{\ell }|u_{\ell }|),$$

$$\underset{ϵ\to 0}{\lim }\frac{{\lambda }_h(ϵA)}{\lambda (ϵA)}=\exp (-\Vert h{\Vert }_{B_1^s}).$$

$$\underset{ϵ\to 0}{\lim }\frac{\lambda (B_{ϵ}(z_1))}{\lambda (B_{ϵ}(z_2))}=\exp (-\Vert z_1{\Vert }_{B_1^s}+\Vert z_2{\Vert }_{B_1^s}),$$

$$y=\mathcal{G}(u)+\xi .$$

Full text

Our key result is to show that in this non-parametric setup the maximum a posteriori (MAP) estimates are characterized by the minimizers of a generalized Onsager–Machlup functional of the posterior. This is done independently for the so-called weak and strong MAP estimates, which as we show coincide in our context. In addition, we prove a form of weak consistency for the MAP estimators in the infinitely informative data limit. Our results are remarkable for two reasons: first, the prior distribution is non-Gaussian and does not meet the smoothness conditions required in previous research on non-parametric MAP estimates. Second, the result analytically justifies existing uses of the MAP estimate in finite but high dimensional discretizations of Bayesian inverse problems with the considered Besov priors.

\startlocaldefs\endlocaldefs

and

1 Introduction

We consider the inverse problem of recovering an unknown functional parameter $u\in X$ from a noisy and indirect observation $y\in Y.$ We work in a framework in which $X$ is an infinite-dimensional separable Banach space, while $Y={\mathbb{R}}^{J}$. In particular, we consider the additive noise model

[TABLE]

where $\xi$ is mean zero Gaussian observational noise, with a positive definite covariance matrix $\Sigma\in{\mathbb{R}}^{J\times J}$, $\xi\sim N(0,\Sigma).$ Here the possibly nonlinear operator $\mathcal{G}:X\to Y$ describes the system response, connecting the observation $y$ to the unknown parameter $u$. More specifically, $\mathcal{G}$ captures both the forward model and the observation mechanism and is assumed to be known.

Inverse problems are mathematically characterized by being ill-posed: the lack of sufficient information in the observation prohibits the unique or stable reconstruction of the unknown. This can either be due to an inherent loss of information in the forward model, or due to the incomplete and noisy observation. To address ill-posedness, regularization techniques are employed in which the available information in the observation is augmented using a priori available knowledge on the properties of candidate solutions.

We adopt a Bayesian approach to the regularization of inverse problems, which has in recent years attracted enormous attention in the inverse problems and imaging literature: see the early work [20], the books [54, 27] and the more recent works [36, 51, 10] and references therein. In this approach, prior information is encoded in the prior distribution $\mu_{0}$ on the unknown $u$, and the Bayesian methodology is used to formally obtain the posterior distribution $\mu^{y}$ on $u|y$, in the form

[TABLE]

where

[TABLE]

Our work is driven by two types of a priori information: on the one hand, we aim to recover unknown functions with a blocky structure, as is typically the case in image processing [16] and medical imaging applications [15]. On the other hand, we are interested in prior models which promote sparse solutions, that is high-dimensional solutions that can be represented by a small number of coefficients in an appropriate expansion. To achieve these effects, we utilize so-called Besov space priors [36, 11], which are well-known for their edge-preserving and sparsity-promoting properties, see e.g. [34, 44, 39, 26, 8]. With these goals in mind, our work relates to classical $L^{1}$-type regularization methods such as penalized least squares with Total Variation penalty [46, 50].

In practice, the implementation of the Bayesian approach to an inverse problem typically requires a high-dimensional discretization of the unknown and large computational resources. From a computational perspective, it is hence imperative that the probability models and estimators related to finite-dimensional discretizations of the problem scale well with respect to refining discretization. In particular, there is a fundamental need to understand whether specialized prior information used frequently in applications, for example of the type described above, leads to well-defined non-parametric probability models and whether the related finite-dimensional estimators have well-behaving limits; this motivates the study of Bayesian inverse problems in the infinite-dimensional function-space setting. A body of literature on this topic has emerged during the last years, centered around two theories:

1. a)

the discretization invariance theory [37, 36], which aims to ensure that when investing more resources in increasing the discretization level, the prior remains faithful to the intended information on the unknown function and the posterior converges to a well defined limit which is an improved representation of the reality; 2. b)

the well-posedness of the posterior theory found in [51, 10], which secures that the posterior is well defined in the infinite-dimensional limit and robust with respect to perturbations in the data as well as approximations of the forward model.

Our work studies Bayesian inversion in function spaces especially from the perspective of point estimators. Namely, we study and give a rigorous meaning to maximum a posteriori (MAP) estimates for Bayesian inverse problems with certain Besov priors in the infinite-dimensional function-space setting. A MAP estimate is understood here as the mode of the posterior probability measure, hence our results require a careful definition of a mode in the infinite-dimensional setting. The main challenge is then to establish a connection between the topological definition of a MAP estimate (mode of the posterior) and an explicit variational problem. Succeeding in doing so, opens up the possibility of studying the behaviour of MAP estimators in certain situations. In particular, we are able to prove a weak form of consistency of the MAP estimator in the infinitely informative data limit.

1.1 Non-gaussian prior information and the need for MAP estimators

A major challenge in Bayesian statistics is the extraction of information from the posterior distribution. In Gaussian-conjugate settings, such as linear inverse problems, on the one hand there are explicit formulae for the posterior mean which can be used as an estimator of the unknown, and on the other hand one can (in principle) sample directly from a discretized version of the posterior, [40, 38, 1, 42, 41]. Nevertheless, draws from Gaussian priors do not vary sufficiently sharply on their domain to have a blocky structure, neither are they sparse.

For this purpose, the so-called TV prior has been used widely in applied literature, e.g. in medical imaging [49, 35]. Drawing intuition from the classical regularization literature, the TV prior has a formal density of the form

[TABLE]

where $\alpha>0$ is a (hyper)parameter. Here, the norm of bounded variation, $\left\|u\right\|_{BV}$, can be formally thought of as the $L^{1}$-norm of the derivative of $u$. However, the numerical implementation of Bayesian inversion with a TV prior miss-behaves as the discretization level of the unknown increases, and in particular the TV prior is not discretization invariant, [37, 36, 34]. For example, depending on the choice of the parameter $\alpha$ as a function of the discretization level, the posterior mean either diverges or converges to an estimate corresponding to a Brownian bridge prior. For alternative types of approaches to edge-preserving non-parametric Bayesian inversion, see [17, 24, 22].

We consider the family of Besov-priors which were proposed and shown to be discretization invariant in [36]. A well-posedness theory of the posterior was developed in [11]. These priors are defined by a wavelet expansion with random coefficients, motivated by a formal density of the form

[TABLE]

where $\left\|\cdot\right\|_{B_{p}^{s}}=\left\|\cdot\right\|_{B_{pp}^{s}}$ is the Besov space norm with regularity parameter $s$ and integrability parameters $p$. For $p=2$ the Besov space $B_{2}^{s}$ corresponds to the Sobolev space of functions with $s$ square-integrable derivatives, $H^{s}$, and the corresponding family of priors are Gaussian with Sobolev-type smoothness parametrized by $s$. We are especially interested in the case $p=1$, which is highly interesting for edge-preserving and sparsity-promoting Bayesian inversion, [34]. For the case $s=1$ this is due to the close resemblance between the way $\left\|u\right\|_{B_{1}^{1}}$ and $\left\|u\right\|_{BV}$ work, both dealing with the $L^{1}$-norm of a (generalized) derivative of $u$, see [34, Section 2]. We define rigorously the class of Besov priors for $p=1$ and $s\in{\mathbb{R}}$, called $B^{s}_{1}$-Besov priors, in section 3 below.

To probe the posterior in the non-conjugate context of linear or nonlinear inverse problems with Besov priors, one typically resorts to Markov chain Monte Carlo (MCMC) methods. Unfortunately, in practice standard MCMC algorithms become prohibitively expensive for large scale inverse problems, consider e.g. photo-acoustic tomography [43, 55]. This is due to the heavy computational effort required for solving the forward problem which is needed for computing the acceptance probability at each step of the chain.

In such situations maximum a posteriori estimates are computationally attractive as they only require solving a single optimization problem. Furthermore, it was shown in [34] that for the $B^{1}_{1}$-Besov prior defined via wavelet bases, at finite discretization levels the resulting MAP estimators are sparse and preserve the locations of the edges in the unknown. This remains true as discretization is refined, while our work validates that, unlike in the TV prior case, the discretized MAP estimators converge to the MAP estimators of the limiting infinite dimensional posterior distribution.

1.2 Review of the main results

Our results draw inspiration from previous papers by the authors [12, 23] (see also [17]), where concepts that we will call strong and weak MAP estimates were coined. We quote both definitions in section 2 below.

As mentioned earlier, we can formally think of a MAP estimator as a mode of the posterior. In finite dimensional contexts, especially when working with continuous probability distributions, the definition of a mode is straightforward as a maximizer of the probability density function. If the prior has probability density function of the form

[TABLE]

for a suitable positive function $W:X\to[0,\infty)$, the density of the posterior is

[TABLE]

where

[TABLE]

In this case, a MAP estimator can be interpreted as a classical estimator of the unknown, arising from Tikhonov regularization with penalty term given by the negative log-density of the prior, [19].

In the infinite dimensional setting things are less straightforward due to the lack of a uniform reference measure. An intuitive approach to define a mode in a function space $X$ is as follows: compute the measure of balls with any center $u\in X$ and a fixed radius $\epsilon>0$ and proceed by letting $\epsilon$ tend to zero. A mode $\hat{u}$ is a center point maximizing these small ball probabilities asymptotically (as $\epsilon$ decreases) in a specific sense. What distinguishes a strong mode from a weak mode is exactly how ‘asymptotic maximality’ is perceived:

1. a)

in the strong mode case, we look for the maximum probability among all centres in $X$; 2. b)

in the weak mode case, we look for a centre of a ball with the property that it has maximum probability among all shifts of the ball by elements of a dense subspace $E\subset X$.

Natural choices of $E$ turn out to be spaces of zero probability, thus giving one interpretation for the term ‘weak’. Note that weak and strong modes coincide for $E=X$. Each of the two notions of mode gives rise to a notion of MAP estimator termed strong and weak MAP estimators, respectively.

Let $B_{\epsilon}(z)\subset X$ denote the open ball of radius $\epsilon$, centered at $z\in X$. If we can find a functional $I$ defined on an appropriate dense subspace $F\subset X$, such that

[TABLE]

when $\mu=\mu^{y}$, then for any fixed $z_{1}\in F$, a $z_{2}\in F$ maximising the limit or equivalently minimising $I$ is a potential MAP estimator. For weak MAP estimators, if the space $E\subset F$ over which we shift the ball centres, can be chosen sufficiently regular so that

[TABLE]

again for $\mu=\mu^{y}$, then it is straightforward to establish the equivalence of weak MAP estimators and the minimisers of $I$. In the case of strong MAP estimators one needs to work considerably more, the difficulty stemming from the ‘smallness’ of the subspace $F\subset X$ with respect to the prior hence also the posterior.

If a functional $I$ satisfying (5) exists, it is called the (generalised) Onsager-Machlup functional, [18, 25, 12, 23]. As for the limit in (6), it is identified with $R^{\mu}_{h}$, the Radon-Nikodym derivative of the shift of $\mu$ by $h\in E$ with respect to $\mu$ itself, provided this derivative has a continuous representative, see [23, lemma 2] (quoted in lemma 2.3 below). In finite dimensions, both the existence of the Onsager-Machlup functional and the statement in (6) follow from the Lebesgue differentiation theorem under very mild conditions on $\Phi$ and the density of the prior. In the Banach-space setting, establishing (5) and (6) for the posterior boils down to showing similar results for the prior given that the posterior is obtained by a suitably regular transformation of the prior.

Strong MAP estimators were defined and studied in [12], in the context of nonlinear Bayesian inverse problems with Gaussian priors. In this case, an expression for the limit in (5) was readily available from the Gaussian literature for $F$ being the Cameron–Martin space of $\mu_{0}$. The identification of minimisers of the resulting Onsager–Machlup functional with strong MAP estimators, however, required a considerable amount of work, in particular many new estimates involving small ball probabilities under the prior.

Weak MAP estimators were defined and studied in [23], in the context of linear Bayesian inverse problems with a general class of priors. The authors used the tools from the differentiation and quasi-invariance theory of measures, developed by Fomin and Skorohod [5], to connect the zero points of $\beta_{h}^{\mu}$, the logarithmic derivative of a measure $\mu$ in the direction $h$, to the minimisers of the Onsager-Machlup functional. An essential assumption that makes this possible is the continuity of $\beta_{h}^{\mu}$ over $X$ for sufficiently regular $h$. The authors considered as examples Besov priors with integrability parameter $p>1$ and a conditionally Gaussian hierarchical prior with a hyper-prior on the mean.

In both [12] and [23], the Onsager-Machlup functional was shown to be a Tikhonov-type functional as in (4), where $W$ is the formal negative log-density of the prior, hence MAP estimators are identified with the corresponding Tikhonov approximations in the studied contexts.

In this work, we are interested in defining and studying both the strong and the weak MAP estimates for generally nonlinear inverse problems with $B^{s}_{1}$-Besov priors, that is for Besov priors with integrability parameter $p=1$. Since the prior has formal density as in (3), we expect and indeed show that MAP estimates are identified with minimizers of the Tikhonov-type functional

[TABLE]

In particular, the weak and strong MAP estimates coincide. For the considered prior, the general theory developed in [23] for weak MAP estimators does not apply, due to the fact that the logarithmic derivative of the $B^{s}_{1}$-Besov priors is inherently discontinuous. We will show that the continuity of the Radon-Nikodym derivative $R^{\mu}_{h}$, for $h$ in a suitable subspace $E\subset X$, is sufficient to get the result for weak MAP estimators. For the $B^{s}_{1}$-Besov priors, we prove the continuity of $R^{\mu}_{h}$ for shifts $h\in E\subseteq B^{r}_{1}(\mathbb{T}^{d})$ for any $r>s$, and establish the validity of (5) over $F=B^{s}_{1}(\mathbb{T}^{d})$. It is then straightforward to prove our first main result which is the identification of weak MAP estimators by the minimisers of the functional $I$ in (7). Furthermore, we generalise the program of [12] to the assumed non-Gaussian case, in order to prove our second main result which is the identification of strong MAP estimators to minimisers of $I$ in (7). We also consider the theory of local weak modes separately, relying on the Fomin literature.

1.3 Consistency of MAP estimators

Under the frequentist assumption of data generated from a fixed underlying true $u^{\dagger}$, it is desirable to verify that in the infinitely informative data limit, the Bayesian posterior distribution $\mu^{y}$ contracts optimally to a Dirac distribution centered on $u^{\dagger}$. In recent years, there have been many studies on the rates of posterior contraction in the context of Bayesian inverse problems. The case of linear inverse problems with Gaussian and conditionally Gaussian priors is now well understood [32, 1, 33, 2, 29, 53, 31], and a theory for linear inverse problems with non-Gaussian priors is also being developed [45, 30]. For nonlinear inverse problems the asymptotic performance of the posterior is not yet fully understood, with some partial contributions being [42, 56, 57].

Note that for general nonlinear problems, especially with finite dimensional data as in the present paper, one cannot expect to recover the underlying truth $u^{\dagger}$ in the infinitely informative data limit. Instead the aim is to recover a $u^{\ast}\in X$ such that $\mathcal{G}(u^{\ast})=\mathcal{G}(u^{\dagger})$. Posterior contraction rates for Bayesian inverse problems with Besov priors are studied in ongoing work of a subset of the authors. A form of weak consistency of the strong MAP estimator in the presence of repeated independent observations, for general nonlinear inverse problems with Gaussian priors, was shown in [12]. In the present paper, we prove a similar result for the strong (hence the weak) MAP estimator obtained using the Besov prior for $p=1$.

1.4 Notation

Throughout the paper we assume that $X$ is a separable Banach space equipped with the Borel $\sigma$-algebra. All probability measures are assumed to be Borel measures. The Euclidean norm in ${\mathbb{R}}^{J}$ is denoted by $|\cdot|$ to distuinguish it from the norm of any general $X$ denoted by $\left\|\cdot\right\|_{X}$. We write $f\propto g$ for two functions $f,g:X\to{\mathbb{R}}$ if there exists a universal constant $c\in{\mathbb{R}}$ such that $f=cg$ as functions.

Definition 1.1**.**

A measure $\mu$ is called quasi-invariant along $h$, if the translated measure $\mu_{h}(\cdot):=\mu(\cdot-h)$ is absolutely continuous with respect to $\mu$. We define

[TABLE]

which is readily verified to be a linear subspace.

Notation 1.2**.**

Let $h\in Q(\mu)$. We denote the Radon–Nikodym derivative of $\mu_{h}$ with respect to $\mu$ by $R_{h}^{\mu}\in L^{1}(\mu)$.

1.5 Organization of the paper

This paper is organized as follows: in section 2 we discuss the definition of modes for probability measures on separable Banach spaces. We introduce novel localized versions of the modes studied in previous work and discuss briefly how different modes can be characterized for log- or quasi-concave measures. The Besov priors are introduced and discussed in section 3 including the key results relating to the Radon–Nikodym derivative of the Besov prior in section 3.2. Section 4 covers the Bayesian inverse problem setup and our main results related to identification of weak and strong MAP estimates as the minimizers of certain variational problem. Moreover, the weak consistency result is given in section 4.2. In section 5 we discuss the logarithmic derivative of the posterior and its use in characterizing the MAP estimates. Finally, all proofs are postponed to section 6.

2 Modes of measures on Banach spaces

In subsection 2.1 we introduce the two existing notions of maximum a posteriori estimator (modes of the posterior measure) proposed in [12, 23] in the context of measures on infinite-dimensional spaces. We also define two new notions of local modes and hence local MAP estimates. In subsection 2.2, we focus on log-concave measures, study the structure of the set of modes, and give conditions for local modes to be global.

2.1 Weak and strong, global and local

The following definition of a mode, introduced in [12], grows out of the idea that highest small ball probabilities are obtained asymptotically at the mode.

Definition 2.1**.**

Let $M^{\epsilon}=\sup_{u\in X}\mu(B_{\epsilon}(u))$. We call a point $\hat{u}\in X$ a mode of the measure $\mu$, if it satisfies

[TABLE]

A mode of the posterior measure $\mu^{y}$ in (2), is called a maximum a posteriori (MAP) estimate.

Below we occasionally use the terms strong mode and strong MAP estimator for the concepts introduced in definition 2.1 in order to distinguish them from the following weaker notion of a mode (similarly, the weak mode or weak MAP) introduced in [23].

Definition 2.2**.**

Let $E$ be a dense subspace of $X$. We call a point $\hat{u}\in X$, $\hat{u}\in{\rm supp}(\mu)$, a weak mode of $\mu$ if

[TABLE]

for all $h\in E$. A weak mode of the posterior measure $\mu^{y}$ in (2), is called a weak maximum a posteriori (wMAP) estimate.

Notice that the definition of a weak mode is dependent on the choice of the subspace $E$. Therefore, in some contexts it may be more appropriate to discuss $E$-weak modes. In the following, however, we will suppress this dependence since the key question to our study is whether such a space exists.

The notions of weak and strong mode are related as follows: any strong mode is a weak mode for the choice $E=X$ [23, lemma 3], which is straightforward to see by simply estimating $\mu(B_{\epsilon}(\hat{u}-h))\leq M^{\epsilon}$. The key motivation to study the weak definition, is the case when small ball asymptotics are not available explicitly or only available in some subspace of translations $h$. It is then of interest to choose $E$ so that an expression for the limit on the left hand side of (8) exists pointwise. This typically leads to choices of $E$ which have zero probability with respect to $\mu$. The following lemma, which is an immediate generalization of [23, lemma 2], provides further guidance for this choice.

Lemma 2.3**.**

Assume that $\mu$ is quasi-invariant along the vector $h$. Let $A\in\mathcal{B}(X)$ be convex, bounded and symmetric and define $A^{\epsilon}:={\epsilon}A$. Suppose $R_{h}^{\mu}$ has a continuous representative $\tilde{R}_{h}^{\mu}\in C(X)$, i.e., $R_{h}^{\mu}-\tilde{R}_{h}^{\mu}=0$ in $L^{1}(\mu)$. Then it holds that

[TABLE]

for any $u\in X$.

Remark 2.4**.**

According to lemma 2.3, it is desirable to consider a subspace $E$, such that $R_{h}^{\mu}$ is continuous for $h\in E$. A sufficient condition for the continuity of $R_{h}^{\mu}$ was given in [23], namely the so-called logarithmic derivative of $\mu$ along $h$ needed to be continuous and exponentially integrable with respect to $\mu$. For the sparsity promoting measure that we will consider in this paper, the logarithmic derivative is inherently discontinuous (section 5). We are however able to show the continuity of $R_{h}^{\mu}$ over an appropriate subspace $E$ in subsection 3.2 by using its explicit expression.

Both strong and weak mode can also be associated with a natural localization described by the following definitions:

Definition 2.5** (Local modes).**

Let $\hat{u}\in X$ be such that $\hat{u}\in{\rm supp}(\mu)$.

1. (1)

We call $\hat{u}$ a local mode of the measure $\mu$, if there exists a $\delta>0$ such that the quantity $M^{\epsilon}_{\delta}=\sup_{u\in B_{\delta}(\hat{u})}\mu(B_{\epsilon}(u))$ satisfies

[TABLE] 2. (2)

We call $\hat{u}$ a local weak mode of $\mu$ if there exists $\delta>0$ such that

[TABLE]

for all $h\in B^{X}_{\delta}(0)\cap E$.

Local modes represent an analogue to local maxima of the probability density function in finite dimensions. In the setting of [23] the local wMAP coincides with the zero points of the logarithmic derivative of the posterior. Especially, regularization techniques in non-linear inverse problems are often known to give birth to local maxima and, therefore, the local wMAP can give statistical interpretation for these points.

2.2 Modes for log-concave measures

This work studies the Besov prior which is a prototypical example of a log-concave measure. Below we show some general properties regarding the modes of such class of measures. Most of these ideas naturally extend to a larger class called the quasi-concave measures.

Definition 2.6**.**

A probability measure $\mu$ in $(X,\mathcal{B}(X))$ is called logarithmically-concave or log-concave, if

[TABLE]

for all $A,B\in\mathcal{B}(X).$ Moreover, $\mu$ is called quasi-concave if

[TABLE]

for all $A,B\in\mathcal{B}(X).$

It is straightforward to verify that any log-concave measure is also quasi-concave. An immediate result of quasi-concavity is the well-known Anderson inequality [3, 7].

Proposition 2.7**.**

Let $\mu$ be a symmetric quasi-concave measure on $X$. For any symmetric and convex set $A\subset X$ we have

[TABLE]

The next result follows from the Anderson inequality.

Proposition 2.8**.**

Suppose that the measure $\mu$ on $X$ is symmetric around $u$ and quasi-concave. Then $u$ is a strong mode of $\mu$.

Let us next consider briefly the structure of the set of strong modes of a quasi-concave measure $\mu$. When working in finite dimensions, $X={\mathbb{R}}^{d}$, a probability density function $f$ with respect to the Lebesgue measure is called quasi-concave if for all $x,y\in{\mathbb{R}}^{d}$ and all $\lambda\in[0,1]$ we have

[TABLE]

Clearly a quasi-concave probability density function has a convex set of global modes. For a reference on convexity and unimodality in finite dimensions see [14]. We show that a similar result holds in infinite dimensions for our definition of strong mode.

Proposition 2.9**.**

Suppose that the measure $\mu$ in $X$ is quasi-concave (but not necessarily symmetric). Then the set of strong modes is convex.

It turns out log-concavity is a sufficient condition for the global and local modes to coincide.

Theorem 2.10**.**

Suppose $\mu$ is log-concave and $\hat{u}$ is a local mode. Then $\hat{u}$ is also a global mode. Similarly, if $\hat{u}$ is a local weak mode then it is also a global weak mode.

3 Besov priors with $p=1$

The family of Besov priors has been introduced in [36] and studied in [11, 23]. In subsection 3.1 we recall the definition and some useful properties of Besov priors with integrability parameter $p=1$ and regularity parameter $s>0$, termed $B^{s}_{1}$-Besov priors, on which we focus in this work. We also present some straightforward convexity properties of $B^{s}_{1}$-Besov priors. The main results of this section are listed in subsection 3.2, where we compute the Radon-Nikodym derivative $R_{h}^{\mu}$ for $B^{s}_{1}$-Besov priors, determine the space $E$ in which $h$ needs to live in order for the corresponding Radon-Nikodym derivative $R_{h}^{\mu}$ to be continuous, and finally show that $I_{0}(u)=\|u\|_{B^{s}_{1}}$ is the Onsager-Machlup functional for $B^{s}_{1}$-Besov measures.

3.1 Definition and basic properties

We work with periodic functions on a $d$-dimensional torus, $\mathbb{T}^{d}$. We first define the periodic Besov spaces $B^{s}_{pq}(\mathbb{T}^{d})$, where $s\in{\mathbb{R}}$ parametrises smoothness and $p,q\geq 1$ are integrability parameters. We concentrate on the case $p=q$ and write $B^{s}_{p}=B^{s}_{pp}$. To define the Besov spaces, we let $\{\psi_{\ell}\}_{\ell=1}^{\infty}$ be an orthonormal wavelet basis for $L^{2}(\mathbb{T}^{d})$, where we have utilized a global indexing. We can then characterise $B^{s}_{p}(\mathbb{T}^{d})$ using the given basis in the following way: the function $f:\mathbb{T}^{d}\to{\mathbb{R}}$ defined by the series expansion

[TABLE]

belongs to $B^{s}_{p}(\mathbb{T}^{d})$, if and only if the norm

[TABLE]

is finite. Throughout, we assume that the basis is $r$-regular for $r$ large enough in order to consist a basis for a Besov space with smoothness $s$, [13].

We now follow the construction in [36] to define periodic Besov priors corresponding to $p=1$, using series expansions in the above wavelet basis with random coefficients. Notice also the work [21] on defining Besov priors for functions on the full space ${\mathbb{R}}^{d}$.

Definition 3.1**.**

Let $(X_{\ell})_{\ell=1}^{\infty}$ be independent identically distributed real-valued random variables with the probability density function

[TABLE]

Let $U$ be the random function

[TABLE]

Then we say that $U$ is distributed according to a $B^{s}_{1}$-Besov prior.

The next lemma determines the smoothness of functions drawn from the $B^{s}_{1}$-Besov prior and shows the existence of certain exponential moments.

Lemma 3.2**.**

[36*, lemma 2]**

Let $U$ be as in Definition 3.1 and let $t<s-d$. Then it holds that*

• (i)

$\left\|U\right\|_{B^{t}_{1}}<\infty$,  almost surely, and

• (ii)

$\mathbb{E}\exp(\frac{1}{2}\left\|U\right\|_{B^{t}_{1}})<\infty$.

Notation 3.3**.**

We denote by $\rho_{\ell}$ the probability measure of the random variable $\ell^{-s/d-1/2}X_{\ell}$ on ${\mathbb{R}}$. We identify the random function $U$ in definition 3.1 with the product measure of coefficients $(\ell^{-s/d-1/2}X_{\ell})_{\ell}$ in $({\mathbb{R}}^{\infty},{\mathcal{B}}({\mathbb{R}}^{\infty}))$, which we denote by $\lambda=\bigotimes_{\ell=1}^{\infty}\rho_{\ell}$.

We next consider the convexity of the $B^{s}_{1}$-Besov prior.

Lemma 3.4**.**

For any $s>0$, the $B^{s}_{1}$-Besov measure $\lambda$ is logarithmically concave.

An immediate consequence of the last lemma, is that by proposition 2.7, the $B^{s}_{1}$-Besov prior satisfies Anderson’s inequality. Notice that lemma 3.4 (and hence proposition 2.7) also holds for $B^{s}_{p}$-Besov measures with $p>1$, as defined in [36].

3.2 Radon–Nikodym derivative $R^{\mu}_{h}$ and small ball probabilities

Recall the definitions of quasi-invariance for a measure $\mu$ and of the subspace of directions in which $\mu$ is quasi-invariant, $Q(\mu)$. In general the structure of $Q$ is not known and there are even examples of measures for which $Q$ fails to be locally convex [5, exercise 5.5.2]. The space $Q$ is known to be a Hilbert space for certain families of measures, for example for $\alpha$-stable measures with $\alpha\geq 1$ and for countable products of a single distribution with finite Fisher information, see [5, theorem 5.2.1] and [48] (note that the $B^{s}_{1}$-Besov prior, $\lambda$, is not an $\alpha$-stable measure). Using similar techniques to [48], namely the Kakutani–Hellinger theory, we now show that $Q$ is a Hilbert space also for the $B^{s}_{1}$-Besov measure $\lambda$ and calculate the Radon-Nikodym derivative $R_{h}^{\lambda}$ between $\lambda$ and the shifted measure $\lambda_{h}$ for $h\in Q(\lambda)$.

Lemma 3.5**.**

For the $B^{s}_{1}$-Besov measure we have $Q(\lambda)=B_{2}^{s-\frac{d}{2}}(\mathbb{T}^{d})$. For $h\in Q(\lambda)$ we have

[TABLE]

in $L^{1}({\mathbb{R}}^{\infty},\lambda)$, where $\alpha_{\ell}=\ell^{s/d-1/2}$.

We next provide a more detailed view of spaces of shifts $h$ for which $R_{h}^{\lambda}$ has a continuous representative, which we denote by $\tilde{R}_{h}^{\lambda}(u)$. This is a crucial result for our study of weak MAP estimators, see lemma 2.3, Remark 2.4 and the discussion in subsection 1.2.

Lemma 3.6**.**

Let $h=\sum_{\ell\in{\mathbb{N}}}h_{\ell}\psi_{\ell}\in B^{r}_{1}(\mathbb{T}^{d})$ with $r>s$. Then \tilde{R}_{h}^{\lambda}(u)=\exp\sum_{\ell=1}^{\infty}\big{(}-\alpha_{\ell}|h_{\ell}-u_{\ell}|+\alpha_{\ell}|u_{\ell}|\big{)} is continuous with respect to $u=\sum_{\ell\in{\mathbb{N}}}u_{\ell}\psi_{\ell}\in B^{t}_{1}(\mathbb{T}^{d})$ for any $t<s-d$.

Note that in the expression for the Radon-Nikodym derivative $\tilde{R}_{h}^{\lambda}(u)$, $h\in E$ and the less regular $u\in X$ are coupled component-wise (in the wavelet basis defining the Besov measure) and hence establishing the continuity of $\tilde{R}_{h}^{\lambda}(u)$ with respect to $u$ is not straightforward. See section 6 for the proof of above lemma.

We record the following immediate corollary of the last lemma and lemma 2.3.

Corollary 3.7**.**

Let $h\in B^{r}_{1}(\mathbb{T}^{d})$, $r>s$. Then it holds that

[TABLE]

for any $u\in B_{1}^{t}$.

Remark 3.8**.**

Let $X=B^{t}_{1}(\mathbb{T}^{d})$ for $t<s-d$. By remark 2.4 and noting that the Besov space $B^{r}_{1}(\mathbb{T}^{d})$ is dense in $B^{t}_{1}(\mathbb{T}^{d})$ for any $r>t$, the last corollary shows that it is natural to choose $E=B^{r}_{1}(\mathbb{T}^{d})$ for any $r>s$ in the definition of wMAP estimate. It also shows that the origin is the unique weak mode and, therefore, by proposition 2.8 also the unique strong mode.

We also remark that the series above consists of the difference of two terms, whose respective series are not convergent in general for $h\in B^{r}_{1}(\mathbb{T}^{d})$. However, they coincide with the difference of the norms in $h\in B^{r}_{1}(\mathbb{T}^{d})$ if $h$ and $u$ are elements of this smaller space. Based on this view we close this section with an important building block for the study of the MAP estimate. It extends the last corollary to $h\in B^{s}_{1}(\mathbb{T}^{d})$, for balls centered at the origin.

Theorem 3.9**.**

Suppose that $h\in B^{s}_{1}(\mathbb{T}^{d})$ and $t<s-d$. Let $A\in\mathcal{B}(B^{t}_{1}(\mathbb{T}^{d}))$ denote a convex and zero-centered symmetric and bounded set. Then

[TABLE]

It follows immediately from the above lemma that for $z_{1},z_{2}\in B^{s}_{1}(\mathbb{T}^{d})$,

[TABLE]

giving the Onsager–Machlup functional of $\lambda$. The space $B^{s}_{1}(\mathbb{T}^{d})$ here, is the largest space on which the Onsager–Machlup functional is defined. This is the space $F$ of the discussion in subsection 1.2 in the case of Besov priors.

It is also worth noting the difference between (13) and (14). The centres of the balls in (13) are in $X=B^{t}_{1}(\mathbb{T}^{d})$, the continuity of the right-hand side in $u$ is due to sufficient regularity of the shift $h$. In (14), the centres of the balls are more regular than (13), but the shift is less regular in general, i.e. only in $B^{s}_{1}(\mathbb{T}^{d})$.

4 Main results

We are now ready to present our main results. We first show the existence of weak and strong MAP estimates in Bayesian inverse problems with $B^{s}_{1}$-Besov priors, and identify both with the minimisers of the Onsager–Machlup functional. Using this characterization, we then prove a weak consistency result for MAP estimators.

4.1 Identification of MAP estimates for Bayesian inverse problems

We consider the inverse problem of estimating a function $u\in X=B^{t}_{1}(\mathbb{T}^{d})$ from a noisy and indirect observation $y\in{\mathbb{R}}^{J}$, modelled as

[TABLE]

Here $\mathcal{G}:B^{t}_{1}(\mathbb{T}^{d})\to{\mathbb{R}}^{J}$ is a locally Lipschitz continuous, possibly non-linear operator and $\xi$ is Gaussian observational noise in ${\mathbb{R}}^{J}$, $\xi\sim N(0,\Sigma)$ for a positive definite covariance matrix $\Sigma\in{\mathbb{R}}^{J\times J}$.

We assume that $u$ is distributed according to the $B^{s}_{1}$-Besov measure $\lambda$ defined in Section 3, so that $\lambda(B^{t}_{1}(\mathbb{T}^{d}))=1$ for $t<s-d$. Under the assumption of local Lipschitz continuity of $\mathcal{G}$, it follows that almost surely with respect to $y$ the posterior distribution $\mu^{y}$ on $u|y$, has the following Radon–Nikodym derivative with respect to the prior $\lambda$:

[TABLE]

where

[TABLE]

and $Z(y)$ is the normalization constant. Indeed, local Lipschitz continuity of $\mathcal{G}$ implies measurability of $\Phi(\cdot,y)$ with respect to $\lambda$; it also, together with non-negativity of $\Phi$, gives the finiteness and non-singularity of $Z$. For details see [10].

Following the intuition described in subsection 1.2, we define the Tikhonov-type functional

[TABLE]

The existence of minimizers for $I$ is classical, however, we include the proof for completeness.

Lemma 4.1**.**

The functional $I(\cdot;y)$ in equation (18) has a minimizer $\hat{u}\in B^{s}_{1}(\mathbb{T}^{d})$.

The underpinning of our main results in theorems 4.3 and 4.5 is that the posterior inherits the property of the prior given in lemma 3.6, i.e. there exists a subspace, where the limit of the translated small ball probability ratios has a continuous representative. Moreover, one can show how this limit is connected with functional $I$. The next result follows directly by combining lemmas 2.3, 3.6 and 3.9, and the local Lipschitz continuity of $\Phi$.

Proposition 4.2**.**

Let $A\in\mathcal{B}(B^{t}_{1}(\mathbb{T}^{d}))$ be convex, bounded and symmetric and define $A^{\epsilon}:={\epsilon}A$.

• i)

For any $h\in B^{r}_{1}(\mathbb{T}^{d})$ with $r>s$ the mapping

[TABLE]

is a continuous function of $u\in B^{t}_{1}(\mathbb{T}^{d})$.

• ii)

The Tikhonov-type functional defined in (18) is the generalized Onsager–Machlup functional for the posterior $\mu^{y}$ in (16). That is, for any $z_{1},z_{2}\in B^{s}_{1}(\mathbb{T}^{d})$,

[TABLE]

The main theorems below show that the weak and strong MAP estimates of the posterior $\mu^{y}$ in (16), are identified with minimisers of functional $I$. In particular, the weak and strong MAP estimates coincide for the inverse problems considered here. This is remarkable since it is not known in general under which conditions there exists weak MAP estimates that are not strong.

Theorem 4.3**.**

An element $u\in B^{s}_{1}(\mathbb{T}^{d})$ minimizes $I(\cdot;y)$ if and only if it is a weak MAP estimate for the posterior measure $\mu^{y}$ in (16).

We note that the last result implies the existence of weak MAP estimates by lemma 4.1. We next show the existence of strong MAP estimators, and that any strong MAP estimate is a minimiser of $I$.

Proposition 4.4**.**

Consider the measure $\mu^{y}$ given by (16) and (17) with the $B^{s}_{1}$-Besov prior $\lambda$ and $\mathcal{G}:B^{t}_{1}(\mathbb{T}^{d})\to{\mathbb{R}}^{J}$ locally Lipschitz for $t<s-d$.

• i)

For any $\delta>0$ there exists $z^{\delta}\in B^{t}_{1}(\mathbb{T}^{d})$ satisfying $z^{\delta}=\arg\max_{z\in X}\mu^{y}(A^{\delta}+z)$, where $A^{\delta}:=\delta A$ with $A$ a convex, symmetric and bounded set in $B^{t}_{1}(\mathbb{T}^{d})$.

• ii)

There is a $\bar{z}\in B^{s}_{1}(\mathbb{T}^{d})$ and a subsequence of $\{z^{\delta}\}_{\delta>0}$ which converges to $\bar{z}$ strongly in $B^{t}_{1}(\mathbb{T}^{d})$.

• iii)

The limit $\bar{z}$ is a strong MAP estimator and a minimizer of $I(u;y)$ in equation (18).

In the following theorem we prove that any minimizer of $I$ is a strong MAP estimate and, hence, show the identification of strong MAP estimates and minimizers of $I$ using part (iii) of proposition 4.4.

Theorem 4.5**.**

Suppose that conditions of proposition 4.4 hold. Then the strong MAP estimators of $\mu^{y}$ are characterised by the minimizers of the Onsager–Machlup functional $I$ given in (18).

The proof of theorem 4.3 concerning weak MAP estimates is relatively straightforward and relies on lemma 3.6, i.e., the ability to consider the subspace $B^{r}_{1}(\mathbb{T}^{d})$, where the Radon–Nikodym derivative $R_{h}^{\mu^{y}}$ has a continuous representative. The proof of proposition 4.4 related to strong MAP estimates is more involved and requires a series of technical lemmas (lemma 6.1 to 6.4) which are stated in section 6.3. Our approach is based on developing asymptotic estimates for the small ball probability ratios by projecting the related measures to finite dimensions, where explicit calculations can be carried out.

The difference in difficulty of the proofs related to the two notions of MAP estimates highlights the flexibility of weak MAP estimators. It seems that explicit calculations are typically required for the proof in the case of strong MAP estimates. For practical purposes, it is a highly interesting future task to develop general conditions under which the two MAP estimate concepts coincide.

Remark 4.6**.**

Proposition 4.3.8 in [5] shows that if $\Phi$ is convex in $u$, then since by lemma 3.4 the $B^{s}_{1}$-Besov prior $\lambda$ is logarithmically-concave, the posterior $\mu^{y}$ is also logarithmically-concave and hence quasi-concave. In that case proposition 2.9 shows that the set of modes is convex. The convexity of $\Phi$ depends on the forward operator $\mathcal{G}$: if for example $\mathcal{G}$ is linear then $\Phi$ is convex, however in the general nonlinear case $\Phi$ may be non-convex.

4.2 Weak consistency of the strong MAP

We consider the frequentist setup, in which

[TABLE]

for a fixed underlying value of the unknown functional parameter $u^{\dagger}\in X$. As before, we assume that $\mathcal{G}:X\to{\mathbb{R}}^{J}$ is locally Lipschitz and $\Sigma\in{\mathbb{R}}^{J\times J}$ is a positive definite matrix.

For this set of data and with a $B^{s}_{1}$-Besov prior, $\lambda$, the posterior measure satisfies

[TABLE]

Here also the Lipschitz continuity of $\mathcal{G}$ implies the well-definedness $\mu^{y_{1},y_{2},\dots,y_{n}}$ [10]. Proposition 4.4 then implies that the strong MAP estimator of the above posterior measure is a minimizer of

[TABLE]

Theorem 4.7**.**

Suppose that $\mathcal{G}:B^{t}_{1}(\mathbb{T}^{d})\to{\mathbb{R}}^{J}$ is locally Lipschitz and $u^{\dagger}\in B^{s}_{1}(\mathbb{T}^{d})$. Denote the minimizers of $I_{n}$ given in (20) by $u_{n}$ for each $n\in N$. Then there exists $u^{*}\in B^{s}_{1}(\mathbb{T}^{d})$ and a subsequence of $\{u_{n}\}$ such that $u_{n}\to u^{*}$ in $B^{\tilde{s}}_{1}(\mathbb{T}^{d})$ almost surely for any ${\tilde{s}}<s$. For any such $u^{*}$ we have $\mathcal{G}(u^{*})=\mathcal{G}(u^{\dagger})$.

If $u^{\dagger}$ lives only in $X=B^{t}_{1}(\mathbb{T}^{d})$, and not necessarily in $B^{s}_{1}(\mathbb{T}^{d})$, we can only get the convergence of $\{\mathcal{G}(u_{n})\}$:

Corollary 4.8**.**

Let $\mathcal{G}$ and $u_{n}$, $n\in{\mathbb{N}}$, satisfy the assumptions of theorem 4.7 and suppose that $u^{\dagger}\in B^{t}_{1}(\mathbb{T}^{d})$. Then there exists a subsequence of $\{\mathcal{G}(u_{n})\}_{n\in{\mathbb{N}}}$ converging to $\mathcal{G}(u^{\dagger})$ almost surely.

Theorem 4.7 states that the true solution is identified in the range of $\mathcal{G}$, which is the natural objective also in regularization theory [19]. The full identification of $u^{\dagger}$ is dependent on further properties of $\mathcal{G}$, e.g., injectivity of $\mathcal{G}$ would immediately yield $u^{*}=u^{\dagger}$.

5 Connections to logarithmic derivative

In this section we discuss the logarithmic derivative of the posterior measure. We mainly revisit known results (see e.g. [5]) and also derive the logarithmic derivative of the posterior $\mu^{y}$ given in (16). The intuition behind logarithmic derivative is that it roughly corresponds to the Gâteaux derivative of the posterior potential $I$. If the logarithmic derivative is smooth, then its zero points can determine the weak MAP estimates as shown in [23]. The Besov $B^{s}_{1}$-prior does not meet this criteria due to the discontinuity of its logarithmic derivative at the origin as we show in theorem 5.8. The logarithmic derivative also determines the Radon–Nikodym derivative as recorded in proposition 5.3 below. Moreover, it can be used as a basis of Newton-type algorithms to estimate the weak MAP in case an explicit form of the potential $I$ is not easily accessible, see e.g. Cauchy priors in [52].

Definition 5.1**.**

A measure $\mu$ on $X$ is called Fomin differentiable along the vector $h$ if, for every set ${\mathcal{A}}\in{\mathcal{B}}(X)$, there exists a finite limit

[TABLE]

It is well-known that if $\mu$ is Fomin differentiable along $h$ then the limit $d_{h}\mu$ is a countably additive signed measure on ${\mathcal{B}}(X)$ and has bounded variation [5]. Moreover, $d_{h}\mu$ is absolutely continuous with respect to $\mu$.

We denote the domain of differentiability by

[TABLE]

Definition 5.2**.**

The Radon–Nikodym density of the measure $d_{h}\mu$ with respect to $\mu$ is denoted by $\beta^{\mu}_{h}$ and is called the logarithmic derivative of $\mu$ along $h$.

Proposition 5.3**.**

[5*, prop. 6.4.1]**

Suppose $\mu$ is a Radon measure on a locally convex space $X$ and is Fomin differentiable along a vector $h\in X$. If it holds that $\exp(\epsilon|\beta^{\mu}_{h}(\cdot)|)\in L^{1}(\mu)$ for some $\epsilon>0$, then $\mu$ is quasi-invariant along $h$ and the Radon–Nikodym density $R_{h}^{\mu}$ of $\mu_{h}$ with respect to $\mu$ satisfies the equality*

[TABLE]

Remark 5.4**.**

Recall that as discussed in Remark 2.4, it is desirable to choose $E$ in the definition of wMAP to be a subspace $E\subset X$ such that $R_{h}^{\mu}$, $h\in E$, has a continuous representative $\tilde{R}_{h}^{\mu}$. Therefore, the integral $\int_{0}^{1}\beta^{\mu}_{h}(u-sh)ds$ in (23) has a measurable representative, which is continuous outside the set $\{u\in X\;|\;\tilde{R}_{h}^{\mu}(u)=0\}$. Moreover, a weak mode $\hat{u}$ of $\mu$ can be equivalently defined by condition

[TABLE]

for all $h\in E$.

The construction of the Besov prior in definition 3.1 is a prototypical example of a product measure. By setting $\lambda_{\ell}=\frac{1}{a_{\ell}}\pi_{X}\left(\frac{x}{a_{\ell}}\right)dx$ for $a_{n}=\ell^{-\left(\frac{s}{d}-\frac{1}{2}\right)}$ we can define the probability law of the Besov prior on $({\mathbb{R}}^{\infty},{\mathcal{B}}({\mathbb{R}}^{\infty}))$ by $\lambda=\otimes_{\ell=1}^{\infty}\lambda_{\ell}$. For product measures, the Fomin differentiability calculus reduces to finite dimensional projections in a straightforward manner.

Lemma 5.5**.**

[5*, prop. 3.4.1 (iii)]**

Let $\mu$ be a probability measure on $({\mathbb{R}},{\mathcal{B}}({\mathbb{R}}))$. Then $\mu$ is Fomin differentiable along $h\neq 0$ if and only if has an absolutely continuous density $\pi_{\mu}$ whose derivative satisfies $\pi_{\mu}^{\prime}\in L^{1}({\mathbb{R}})$. In this case, $d_{1}\mu=\pi_{\mu}^{\prime}dx$.*

The Fomin differentiability of a product measure $\mu=\otimes_{n=1}^{\infty}\mu_{n}$ on the space $X=\prod_{n=1}^{\infty}X_{n}$ assigned with the product topology is characterized by the following theorem:

Proposition 5.6**.**

[5*, prop. 4.1.1.]**

Suppose that $\beta^{\mu_{n}}_{h_{n}}$ is the logarithmic derivative of $\mu_{n}$ in the direction $h_{n}\in X_{n}$. The following claims are equivalent:*

• (i)

$\mu$* is differentiable along $h=(h_{j})_{j=1}^{\infty}\in X$,*

• (ii)

the series $\sum_{n=1}^{\infty}\beta_{h_{n}}^{\mu_{n}}$ converges in the norm of $L^{1}(\mu)$ and

[TABLE]

Let us introduce the following subspace

[TABLE]

which has a natural Hilbert space structure [5, section 5]. Surprisingly, for a large class of product measures $H(\mu)$ coincides with $D(\mu)$. The following proposition follows from [6, cor. 2] and [5, ex. 5.2.3].

Proposition 5.7**.**

Suppose $\mu=\pi(x)dx$ is a Borel probability measure on the real line such that

[TABLE]

If we set $\mu_{n}(A)=\mu(A/a_{n})$, where $a_{n}>0$, and $\mu=\otimes_{n=1}^{\infty}\mu_{n}$, then it follows that

[TABLE]

Let us record the following direct consequence of proposition 5.3: if $t>0$ then

[TABLE]

in $L^{1}(\mu)$. It is rather easy to see that $\lambda_{1}$ (and consequently $\lambda_{\ell}$ for any $\ell\in{\mathbb{N}}$) is Fomin differentiable since

[TABLE]

for any ${\mathcal{A}}\in{\mathcal{B}}({\mathbb{R}})$. In more generality, this follows from lemma 5.5 since the density function is absolutely continuous.

Theorem 5.8**.**

Let $\lambda$ be the $B^{s}_{1}$-Besov measure given in definition 3.1. The set of differentiability is given by $D(\lambda)=B^{s-\frac{d}{2}}_{2}(\mathbb{T}^{d})$ and for any $h=\sum_{\ell=1}^{\infty}h_{\ell}\phi_{\ell}\in D(\lambda)$ we have

[TABLE]

where

[TABLE]

is the logarithmic derivative of $\rho_{\ell}$ (see notation 3.3).

Notice that the previous theorem directly states that for any $h\in B_{1}^{s}(\mathbb{T}^{d})$ the logarithmic derivative is bounded $|\beta^{\lambda}_{h}(u)|\leq C\left\|h\right\|_{B_{1}^{s}}$ $\lambda$-almost surely.

For the posterior distribution $\mu^{y}$ we can solve the logarithmic derivative by using properties of the prior and the functional $\Phi$. The following result follows directly from [5, prop. 3.3.12] (see also [17, thm. 5.7]).

Theorem 5.9**.**

Suppose that $\Phi:B^{t}_{1}(\mathbb{T}^{d})\to{\mathbb{R}}$, with $t<s-d$, is bounded from below and possesses a uniformly bounded derivative. Then we have $\beta^{\mu^{y}}_{h}=-\partial_{h}\Phi(u)-\beta^{\lambda}_{h}(u)$ for any $h\in D(\lambda)$.

6 Proofs

6.1 Proofs of results in section 2

Proof of proposition 2.8.

Without loss of generality assume that $\mu$ is symmetric around the origin and show that the origin is a strong mode. For any $u\in X$ the Anderson inequality (proposition 2.7) implies $\mu(B_{\epsilon}(u))\leq\mu(B_{\epsilon}(0))$, and so $\mu(B_{\epsilon}(0))=\sup_{u\in X}\mu(B_{\epsilon}(u))$ and the origin is a strong mode of $\mu$. ∎

Proof of proposition 2.9.

Suppose $u_{1},u_{2}$ are strong modes. For $\kappa\in(0,1)$ we show that $\hat{u}=\kappa u_{1}+(1-\kappa)u_{2}$ is also a strong mode. For $\epsilon>0$ define $M_{\epsilon}=\sup_{u\in X}\mu(B_{\epsilon}(u))$. By quasi-concavity and the identity

[TABLE]

we have that

[TABLE]

so that since $u_{1},u_{2}$ are strong modes we get

[TABLE]

Since for all $\epsilon>0$ we have $\mu(B_{\epsilon}(\hat{u}))/M_{\epsilon}\leq 1$, we get that $\hat{u}$ is a strong mode. ∎

Proof of theorem 2.10.

Let us consider the identity (25) with values $u_{1}=\hat{u}-h$ and $u_{2}=\hat{u}$. By applying log-concavity we have

[TABLE]

and, consequently,

[TABLE]

for any $0\leq\kappa\leq 1$.

Now suppose $\hat{u}\in X$ is local mode in a neighborhood $B_{\delta}(\hat{u})$ but not a global mode. Then there exists $\delta>0$ such that $\hat{u}$ is a local mode in the neighborhood $B_{\delta}(\hat{u})$ but not in $B_{\delta+1}(\hat{u})$. That is, in the larger neighborhood $B_{\delta+1}(\hat{u})$ we have for some $\eta>0$ that there exists such a subsequence $\{\epsilon_{j}\}_{j=1}^{\infty}$ that

[TABLE]

Now let us choose a sequence $\{u_{j}\}_{j=1}^{\infty}\subset B_{\delta+1}(\hat{u})$ that

[TABLE]

Then it follows that

[TABLE]

Since $\hat{u}$ is a local mode there exists $\tilde{\epsilon}>0$ such that for any $\epsilon<\tilde{\epsilon}$ we have

[TABLE]

for $\tilde{\delta}=\frac{\delta}{2(1+\delta)}$ and some $\eta>0$. In fact, for this choice of $\tilde{\delta}$ we have $\hat{u}-\tilde{\delta}(\hat{u}-u_{j})\in B_{\delta}(\hat{u})$ and it follows by (26) for any $\epsilon_{j}<\tilde{\epsilon}$ that

[TABLE]

This yields a contradiction and proves the claim for strong MAP estimates.

Suppose now that $\hat{u}\in X$ is a local wMAP but not a global wMAP. Assume like above that $B_{\delta}(\hat{u})$ is the maximal neighborhood, where $\hat{u}$ is a local wMAP. Then there exists an element $h\in E$ and $h\notin B_{\delta}(\hat{u})$ such that

[TABLE]

for $\tilde{\delta}=\frac{\delta}{2\left\|h\right\|_{X}}$, since $\hat{u}-\tilde{\delta}h\in B_{\delta}(\hat{u})$. Again we see that the inequality (26) yields a contradiction. This completes the proof. ∎

6.2 Proofs of results in section 3

Proof of lemma 3.4.

Let $\lambda^{N}=\bigotimes_{\ell=1}^{N}\rho_{\ell}$. Then it is straightforward to check that $\lambda^{N}$ converges weakly to $\lambda$ as $N\to\infty$. By [7, theorem 2.2], for $\lambda$ to be logarithmically concave, it suffices to show that the measures $\lambda^{N}$ in ${\mathbb{R}}^{N}$ are logarithmically concave. Note that the measures $\lambda^{N}$ have density, denoted $\pi_{N}$, with respect to the Lebesgue measure in ${\mathbb{R}}^{N},$ given by

[TABLE]

for all $x=(x_{1},..,x_{N})\in{\mathbb{R}}^{N}$, where $\alpha_{\ell}=\ell^{\frac{s}{d}-\frac{1}{2}}$ are the coefficients in the expansion defining the $B^{s}_{1}$-Besov measure. Since the density $\pi_{N}$ is a logarithmically concave function, by [4, theorem 1.8.4] $\lambda^{N}$ is logarithmically concave and the result follows. ∎

Proof of lemma 3.5.

We use the Kakutani-Hellinger theory [9, Chapter 2]. We start by calculating the Hellinger integrals $H(\rho_{h,\ell},\rho_{\ell})$, where $\rho_{h,\ell}(\cdot):=\rho_{\ell}(\cdot-h_{\ell})$:

[TABLE]

By [9, lemma 2.5], we have

[TABLE]

where $H_{N}=\prod_{{\ell}=1}^{N}{\rm e}^{-\frac{\alpha_{\ell}}{2}{|h_{\ell}|}}\left(1+\frac{\alpha_{\ell}}{2}{|h_{\ell}|}\right)\in(0,1]$. By taking the negative logarithm we get

[TABLE]

By [9, theorem 2.7] the set $Q(\lambda)$ coincides with the set of $h$ such that $-\log\left(H(\lambda_{h},\lambda)\right)<\infty.$

The Taylor theorem implies the upper and lower bounds

[TABLE]

Using the lower bound, we get that

[TABLE]

which implies that a sufficient condition for the equivalence of $\lambda_{h}$ and $\lambda$ is

[TABLE]

Using the upper bound, and letting $x_{\ell}=\frac{\alpha_{\ell}}{2}|h_{\ell}|$, we get that

[TABLE]

If $x_{\ell}$ is unbounded, then the sum on the right hand side is infinite and we have that $\lambda_{h}$ and $\lambda$ are singular (note that if $x_{\ell}$ is unbounded then obviously condition (27) does not hold). If $x_{\ell}$ is bounded, $x_{\ell}\leq M$, then

[TABLE]

therefore condition (27) is also necessary for the equivalence of the measures $\lambda_{h}$ and $\lambda$.

Note, that condition (27) is equivalent to $h\in B^{s-\frac{d}{2}}_{2}(\mathbb{T}^{d})$. Observing that

[TABLE]

for all $x\in{\mathbb{R}}$ and $\ell\in{\mathbb{N}}$, the claimed expression for $\frac{d\lambda_{h}}{d\lambda}$ follows from [9, theorem 2.7].∎

Proof of lemma 3.6.

We have

[TABLE]

if $|u_{\ell}|\leq|h_{\ell}|$ and

[TABLE]

if $|u_{\ell}|>|h_{\ell}|$. Fix $\eta>0$ and consider $v\in B^{t}_{1}(\mathbb{T}^{d})$ such that $\|v-u\|_{B^{t}_{1}}<\eta$. Let us next define index sets

[TABLE]

Clearly $A_{j}$ are disjoint and ${\mathbb{N}}=\cup_{j=1}^{4}A_{j}$. We utilise the index sets $A_{j}$ to rewrite

[TABLE]

where

[TABLE]

We continue by studying the terms $\mathcal{I}_{j}$ separately. Let $0<\delta<1$ be a value, which we later fix. For $\mathcal{I}_{1}$ we have by the Hölder inequality

[TABLE]

for $\delta_{1}=\frac{\delta}{1-\delta}$, where to bound the second parenthesis in the second to last line we have used that $|v_{\ell}-u_{\ell}|\leq 2|h_{\ell}|$ for any $\ell\in A_{1}$. For $\mathcal{I}_{2}$ we have

[TABLE]

We note that since $|v_{\ell}|>|h_{\ell}|\geq|u_{\ell}|$, the following two inequalities hold:

[TABLE]

As a direct consequence we have

[TABLE]

for any $0\leq\delta\leq 1$. Moreover, since

[TABLE]

we obtain

[TABLE]

where as before $\delta_{1}=\frac{\delta}{1-\delta}$. The term $\mathcal{I}_{3}$ is very similar to $\mathcal{I}_{2}$ (only the role of $u_{\ell}$ and $v_{\ell}$ is swapped), and we have

[TABLE]

For $\mathcal{I}_{4}$ we first note that since $|v_{\ell}|>|h_{\ell}|$ and $|u_{\ell}|>|h_{\ell}|$ for $\ell\in A_{4}$, if $|h_{\ell}|>\frac{\eta}{2\beta_{\ell}}$, with $\beta_{\ell}=\ell^{\frac{t}{d}-\frac{1}{2}},$ we must have ${\rm sign}(u_{\ell})={\rm sign}(v_{\ell})$. Now it follows that

[TABLE]

where

[TABLE]

Combining, since for any $r>s$ there exists $\delta>0$ small enough so that $h\in B^{s+\delta_{1}(s-t)}_{1}(\mathbb{T}^{d})\cap B^{s+\delta_{2}}_{1}(\mathbb{T}^{d})$, we get that as $\eta\to 0$, $\tilde{R}_{h}^{\lambda}(v)-\tilde{R}_{h}^{\lambda}(u)\to 0$. This proves the claim. ∎

Proof of lemma 3.9.

Let $A^{\epsilon}:=\epsilon A$. We first note that by lemma 3.5 we can write

[TABLE]

This implies that

[TABLE]

Now consider $\{h^{j}\}_{j=1}^{\infty}\subset B_{1}^{s+1}(\mathbb{T}^{d})$ with $h^{j}\to h$ in $B^{s}_{1}(\mathbb{T}^{d})$. We have

[TABLE]

This, using lemmas 3.6 and 2.3 implies that

[TABLE]

and letting $j\to\infty$ in the right-hand side gives

[TABLE]

The above inequality together with (31) give the result.

∎

6.3 Proofs of results in section 4

In some of the proofs below we consider the sequences that converge in the weak*-topology of $B^{t}_{1}(\mathbb{T}^{d})$. We note that the Banach space $B^{t}_{1}(\mathbb{T}^{d})$ is isomorphic to a weighted $l^{1}$ space, and hence its pre-dual is the space of functions $\{v\in B^{-t}_{\infty}(\mathbb{T}^{d}):\lim_{j\to\infty}\langle v,\psi_{\ell}\rangle=0\}$.

Proof of lemma 4.1.

Suppose $\{u_{j}\}_{j=1}^{\infty}\subset B^{t}_{1}(\mathbb{T}^{d})$ is a minimizing sequence of functional $I$. Clearly, we can assume $\{I(u_{j},y)\}_{j=1}^{\infty}$, and therefore also $\left\|u_{j}\right\|_{B^{s}_{1}}$ to be bounded. By the Banach–Alaoglu theorem there exists a subsequence that converges to some $\hat{u}\in B^{s}_{1}(\mathbb{T}^{d})$ in the weak*-topology. Notice that the norm of $B^{s}_{1}(\mathbb{T}^{d})$ is lower semicontinuous in the weak*-topology and consequently $\hat{u}\in B^{s}_{1}(\mathbb{T}^{d})$. We now show the strong convergence of the above subsequence in $B^{\tilde{s}}_{1}(\mathbb{T}^{d})$ for any ${\tilde{s}}<s$. We have

[TABLE]

Noting that $\|\hat{u}\|_{B^{s}_{1}}+\|u_{j}\|_{B^{s}_{1}}\leq C<\infty$, given any $\epsilon>0$, $N$ can be chosen large enough, independently of $j$, such that the second term in the last line of the above inequality is bounded by $\epsilon/2$. Having convergence coefficient-wise, there is $M\in{\mathbb{N}}$ large enough so that for $j>M$ the first term is bounded by $\epsilon/2$ as well. We therefore conclude that $u_{j}\to\hat{u}$ in $B^{\tilde{s}}_{1}(\mathbb{T}^{d})$ for any ${\tilde{s}}<s$ and hence in particular for ${\tilde{s}}=t<s-d$. By the continuity assumption on $\Phi$, it now follows that

[TABLE]

Therefore, $\hat{u}$ must be a minimizer. ∎

Proof of theorem 4.3.

Assume that $u_{min}\in B^{s}_{1}(\mathbb{T}^{d})$ is a minimizer of $I(\cdot;y)$. By lemma 3.6 and Lipschitz continuity of $\mathcal{G}$ we know that $R_{h}^{\mu^{y}}\in C(B^{t}_{1}(\mathbb{T}^{d}))$ for any $h\in B^{r}_{1}(\mathbb{T}^{d})$ with $r>s$. Since $u_{min}\in B^{s}_{1}(\mathbb{T}^{d})$, we can study $R_{h}^{\mu^{y}}$ pointwise and obtain

[TABLE]

for any $h\in B^{r}_{1}(\mathbb{T}^{d})$ due to the minimizing property of $u_{min}$. Therefore, $u_{min}$ is a weak MAP.

Consider the reversed claim and assume that $\hat{u}$ is a weak MAP to the posterior $\mu^{y}$ in (16). Let us also assume that $\hat{u}\in B^{t}_{1}(\mathbb{T}^{d})\setminus B^{s}_{1}(\mathbb{T}^{d})$. Due to the continuity of $\Phi$ and lemma 3.6 we have

[TABLE]

for any $h\in B^{r}_{1}(\mathbb{T}^{d})$, $r>s$. Let us construct a particular function $h^{N}=\sum_{\ell=1}^{\infty}h^{N}_{\ell}\psi_{\ell}\in B^{r}_{1}(\mathbb{T}^{d})$ by defining its coefficient vector according to

[TABLE]

for some small $\epsilon>0$. It follows by inequality (32) and continuity of $\Phi$ that

[TABLE]

where $C>0$ is the local Lipschitz constant on the neighbourhood of $\hat{u}$. However, $N$ was chosen arbitrarily and by our assumption on the smoothness of $\hat{u}$ the sum on the left hand side of (33) does not stay bounded when $N$ increases. Therefore, inequality (33) leads to a contradiction and we must have $\hat{u}\in B^{s}_{1}(\mathbb{T}^{d})$.

Assuming now that the weak MAP $\hat{u}\in B^{s}_{1}(\mathbb{T}^{d})$, we can separate the sum in (32) and obtain

[TABLE]

for any $h\in B^{r}_{1}(\mathbb{T}^{d})$. By continuity of $I$ and density of $B^{r}_{1}(\mathbb{T}^{d})$ in $B^{s}_{1}(\mathbb{T}^{d})$, we find that $\hat{u}$ minimizes $I$. ∎

Proof of proposition 4.4 relies on the following four lemmas giving some properties of the Besov prior measure we have here. We list these lemmas and their proofs first.

Lemma 6.1**.**

Let $X$ be a separable Banach space, and $B$ an open and convex set in $\mathcal{B}(X)$. For any non-degenerate measure $\mu$ with full support we have $\mu(\partial B)=0$.

Proof.

For any $\epsilon>0$, there exists a cylindrical set $B_{\epsilon}$ with $B_{\epsilon}\supset B$ satisfying $\mu(B_{\epsilon})-\mu(B)\leq\epsilon$ [4, lemma 2.1.6]. By definition of cylindrical sets, there exists some $n\in{\mathbb{N}}$ and $B_{\epsilon}^{0}\in\mathcal{B}({\mathbb{R}}^{n})$ such that

[TABLE]

Let $h=(l_{1},\dots,l_{n})$. Without loss of generality we can assume that $l_{j}\in X^{*}$, $j=1,...,n$, are linearly independent and, therefore, $h$ is surjective (see discussion in [4, Sec. 2.1.]). Now we have $B\subset h^{-1}(hB)\subset B_{\epsilon}$. Notice carefully that $hB$ is an open set, since $h$ is an open map by the open mapping theorem. Hence we obtain

[TABLE]

By our assumption on non-degeneracy of $\mu$, we have that $\mu_{n}$ is absolutely continuous with respect to the Lebesgue measure in ${\mathbb{R}}^{n}$, and hence we have $\mu_{n}(\partial(hB))=0$. Noting that $\mu(h^{-1}(hB)\setminus B)\leq\mu(B_{\epsilon})-\mu(B)\leq\epsilon$, the result follows. ∎

In the following $\lambda$ stands for a centered $B^{s}_{1}$-Besov prior on $B^{t}_{1}(\mathbb{T}^{d})$ with $t<s-d$. Also, we frequently consider projections of $\lambda$ to a subspace ${\rm span}\{\psi_{1},\cdots,\psi_{n}\}\subset B^{t}_{1}(\mathbb{T}^{d})$. We write $P_{n}:B^{t}_{1}(\mathbb{T}^{d})\to{\mathbb{R}}^{n}$ as

[TABLE]

and define $\lambda_{n}(A):=(\lambda\circ P_{n}^{-1})(P_{n}A)$ for any $A\in{\mathcal{B}}(B^{t}_{1})$.

Lemma 6.2**.**

Let $A\subset\mathcal{B}(B^{t}_{1}(\mathbb{T}^{d}))$ be any convex, symmetric and bounded set with diameter $\delta=\sup_{u,v\in A}\left\|u-v\right\|_{B^{t}_{1}}>0$. For any $z\in B^{t}_{1}(\mathbb{T}^{d})$, with $t<s-d$, we have

[TABLE]

Proof.

First consider the subspace ${\rm span}\{\psi_{1},\cdots,\psi_{n}\}\subset B^{t}_{1}(\mathbb{T}^{d})$ and let $\tilde{\alpha}_{\ell}=\ell^{\frac{t}{d}-\frac{1}{2}}$. Now recall $\alpha_{\ell}\geq\tilde{\alpha}_{\ell}$ since $s>t+d$. We have

[TABLE]

where $\tilde{\lambda}_{n}=\prod_{\ell=1}^{n}\tilde{\rho}_{\ell}$ with $\tilde{\rho}_{\ell}\sim\tilde{c}_{\ell}\exp\left((\alpha_{\ell}-\frac{1}{2}\tilde{\alpha}_{\ell})|u_{\ell}|\right)$. Since $\tilde{\lambda}_{n}$ is logarithmically concave, by theorem 6.1 of [7] (see the proof of proposition 2.7 above) we have that $\tilde{\lambda}_{n}(A+z)\leq\tilde{\lambda}_{n}(A)$.

To consider the limiting case, note that $P_{n}^{-1}(P_{n}A)$ is convex and symmetric. Therefore, we can use the first part of the proof above, and write

[TABLE]

with $\epsilon\to 0$ as $n\to\infty$ due to weak convergence of $\lambda_{n}$ to $\lambda$ and since $A$ is a continuity set by lemma 6.1. ∎

Lemma 6.3**.**

Suppose that $\bar{z}\notin B^{s}_{1}(\mathbb{T}^{d})$, $\{z^{\delta}\}_{\delta>0}\subset B^{t}_{1}(\mathbb{T}^{d})$, $t<s-d$, and $z^{\delta}$ converges to $\bar{z}$ in the weak-topology of $B^{t}_{1}(\mathbb{T}^{d})$ as $\delta\to 0$. Then for any $\epsilon>0$ there exists $\delta$ small enough such that*

[TABLE]

for $A^{\delta}=\delta A$ where $A$ any convex, symmetric and bounded set in $\mathcal{B}(B^{t}_{1}(\mathbb{T}^{d}))$.

Proof.

Below we write $u_{\ell}=\langle u,\psi_{\ell}\rangle$ for any $u\in B^{t}_{1}(\mathbb{T}^{d})$ and without losing any generality assume that $A$ has diameter $1$. Since $\bar{z}\notin B^{s}_{1}(\mathbb{T}^{d})$, for any $M>0$ there is an $N$ large enough such that

[TABLE]

Let $\delta_{0}<2MN^{\frac{t-s}{d}}$. Since $z^{\delta}$ converges to $\bar{z}$ in weak*-topology as $\delta\to 0$, we have $\langle\psi_{\ell},z^{\delta}\rangle\to\langle\psi_{\ell},\bar{z}\rangle$ for all $j$, and therefore $\delta_{1}<\delta_{0}$ can be chosen small enough such that

[TABLE]

Thus, for any $z\in A^{\delta_{1}}+z^{\delta_{1}}$, we can write

[TABLE]

Let $\delta\leq\delta_{1}$ be sufficiently small so that

[TABLE]

For $\lambda_{n}$ we have that for any $M>0$ there exist $N>0$ and $\delta_{1}>0$ such that for $n\geq N$ and $\delta<\delta_{1}$ it follows

[TABLE]

where in the last line we have used the fact that the density in the integrals of the third line is log concave and $A$ is absolutely convex. Similar inequality as (35) generalizes the result for $\lambda$. ∎

Lemma 6.4**.**

Suppose that $\{z^{\delta}\}_{\delta>0}\subset B^{t}_{1}(\mathbb{T}^{d})$ converges in weak-topology and not strongly in $B^{t}_{1}(\mathbb{T}^{d})$ to [math] as $\delta\to 0$. Then for any $\epsilon>0$ there exists $\delta$ small enough such that*

[TABLE]

for $A^{\delta}=\delta A$ where $A$ is any convex, bounded and symmetric set in $\mathcal{B}(B^{t}_{1}(\mathbb{T}^{d}))$.

Proof.

Let $\tilde{\alpha}_{\ell}=\ell^{\frac{t}{d}-\frac{1}{2}}$ and without loss of generality assume that $A$ has diameter $1$. For any $j\in{\mathbb{N}}$ we have $\langle\psi_{\ell},z^{\delta}\rangle\to 0$, as $\delta\to 0$. There exists a subsequence which we relabel $\{z^{\delta}\}$ for which elements there exists $\kappa>0$ such that

[TABLE]

Let $M>0$ be arbitrary and choose $N$ large enough so that $\alpha_{\ell}>M\tilde{\alpha}_{\ell}$ for any $j>N$. By the weak* convergence there exists $\delta$ small enough such that

[TABLE]

where $z^{\delta}_{\ell}=\langle\psi_{\ell},z^{\delta}\rangle$. This then, using (36), means that there exists $n>N$ such that

[TABLE]

Now one can show that

[TABLE]

and

[TABLE]

Having these bounds we obtain

[TABLE]

where we used the log-concavity of the integrands on the last line. Since $\kappa$ is fixed, $M$ is arbitrary and $\delta$ decreases, the result follows for $\lambda_{n}$. In a same way as the previous two lemmas an inequality similar to (35) yields the result. ∎

Proof of proposition 4.4.

i) Without losing any generality we assume that the diameter of $A$ is $1$. The function $z\mapsto\mu^{y}(A^{\delta}+z)$ for fixed $\delta>0$ is bounded from $B^{t}_{1}(\mathbb{T}^{d})$ to $[0,1]$. Let $\{z_{j}\}_{j=1}^{\infty}\subset B^{t}_{1}(\mathbb{T}^{d})$ denote the maximizing sequence such that

[TABLE]

Suppose the sequence $z_{j}$ is unbounded. Since $\Phi\geq 0$ we have for any $z\in B^{t}_{1}(\mathbb{T}^{d})$ that

[TABLE]

by lemma 6.2. Now as $\|z_{j}\|_{B^{t}_{1}}\to\infty$, we have $\mu^{y}(A^{\delta}+z_{j})\to 0$ which yields a contradiction. Therefore, the sequence $z_{j}$ must be bounded.

Next, by the Banach–Alaoglu theorem there exists a limit $w\in B^{t}_{1}(\mathbb{T}^{d})$ in the weak-* topology. In particular, we have $P_{n}z_{j}\to P_{n}w$ in ${\mathbb{R}}^{n}$ for any $n>0$. Let us write $\mu^{y}_{n}(A)=\mu^{y}\circ P_{n}^{-1}(P_{n}A)$ for any $A\in{\mathcal{B}}(B^{t}_{1}(\mathbb{T}^{d}))$. By weak convergence of $\mu^{y}_{n}$ to $\mu^{y}$ for any $\epsilon>0$ there exists $N>0$ such that for $n>N$ we have

[TABLE]

since $A^{\delta}$ is a set of continuity for $\mu^{y}$ by lemma 6.1, i.e. $\mu^{y}(\partial A^{\delta})=0$. In consequence, we have

[TABLE]

where the convergence to $\epsilon$ in (38) appears since $\lambda_{n}$ is non-degenerate. Since $\epsilon>0$ was arbitrary, we obtain

[TABLE]

and by (37) the supremum must be attained at $z^{\delta}=w\in B^{t}_{1}(\mathbb{T}^{d})$.

ii) Since $\Phi(u)\geq 0$ and $\mathcal{G}$ is locally Lipschitz continuous, for any $\delta\leq 1$ we have

[TABLE]

with a constant $L$ independent of $\delta$. Suppose that $\{z^{\delta}\}$ is not bounded in $X$. Then by lemma 6.2, for any $\epsilon>0$, there exists $\delta$ small enough such that

[TABLE]

This contradicts the first inequality in (39). Hence $\{z^{\delta}\}$ is bounded in $B^{t}_{1}(\mathbb{T}^{d})$ for any $t<s-d$ and there exists $\bar{z}\in B^{t}_{1}(\mathbb{T}^{d})$ and a subsequence (also denoted by) $\{z^{\delta}\}\subset B^{t}_{1}(\mathbb{T}^{d})$ such that $z^{\delta}$ converges to $\bar{z}$ in the weak*-topology. Now lemma 6.3 and 6.4 together with (39) imply that $\bar{z}\in B^{s}_{1}(\mathbb{T}^{d})$ and $z^{\delta}\to\bar{z}$ in $B^{t}_{1}(\mathbb{T}^{d})$.

iii) We first note that by local Lipschitz continuity of $\Phi$, there exists a constant $L$ depending on $\|\bar{z}\|_{B^{t}_{1}}$ such that

[TABLE]

and therefore, since $\Phi$ is continuous on $B^{t}_{1}(\mathbb{T}^{d})$ and $z^{\delta}\to\bar{z}$ in $B^{t}_{1}(\mathbb{T}^{d})$, we have

[TABLE]

Now consider a sequence $\{w^{j}\}_{j=1}^{\infty}\subset B^{s+1}_{1}(\mathbb{T}^{d})$ with $w^{j}\to\bar{z}$ in $B^{s}_{1}(\mathbb{T}^{d})$ as $j\to\infty$. Then we have

[TABLE]

This by lemma 3.6 implies that

[TABLE]

and then letting $j\to\infty$ in the right-hand side we get

[TABLE]

Since by definition of $z^{\delta}$ we have that ${\mu^{y}}(A^{\delta}+z^{\delta})\geq{\mu^{y}}(A^{\delta}+{\bar{z}})$, we get that

[TABLE]

It follows that

[TABLE]

and therefore $\bar{z}$ is a strong MAP estimator.

It remains to show that $\bar{z}$ is a minimizer of $I$. Let $z^{*}:=\arg{\min}_{u\in B^{s}_{1}}I(u)$. Suppose that $\bar{z}$ is not a minimizer so that $I(\bar{z})>I(z^{*})$. We first note that by local Lipschitz continuity of $\Phi$, as before we have that there is an $L$ depending on $\|\bar{z}\|_{B^{t}_{1}}$ and $\|z^{*}\|_{B^{t}_{1}}$ such that

[TABLE]

Now consider a sequence $\{w^{j}\}_{j=1}^{\infty}\subset B^{s+1}_{1}(\mathbb{T}^{d})$ with $w^{j}\to\bar{z}$ in $B^{s}_{1}(\mathbb{T}^{d})$. Then, similar to what we did above, we can show that

[TABLE]

as $j\to\infty$. We now note that

[TABLE]

which by (40) gives

[TABLE]

This contradicts the fact that by definition of $z^{\delta}$, $\mu^{y}(A^{\delta}+z^{\delta})\geq\mu^{y}(A^{\delta}+z^{*})$. ∎

Proof of theorem 4.5.

Suppose that $\tilde{z}$ is a strong MAP estimator. Any strong MAP estimate is a weak MAP estimate and hence, by theorem 4.3, $\tilde{z}$ is a minimizer of $I$.

Now let $z^{*}$ be a minimizer of $I$. By proposition 4.4 we know that there exists a strong MAP estimate $\bar{z}\in B^{s}_{1}(\mathbb{T}^{d})$ which also minimises $I$. Therefore, by proposition 4.2, we have

[TABLE]

Let $z^{\epsilon}=\arg\min_{z\in X}\mu^{y}(A^{\epsilon}+z)$. By Definition 2.1, We can write

[TABLE]

The result follows. ∎

Proof of theorem 4.7.

Substituting $y_{j}$ by $\mathcal{G}(u^{\dagger})+\xi_{j}$, we have

[TABLE]

Since $u_{n}$ is a minimizer of $I_{n}$, we can write

[TABLE]

using Young’s inequality. Taking the expectation and using the independence of $\{\xi_{j}\}$, we obtain

[TABLE]

and, by application of the Jensen inequality,

[TABLE]

First, by (41), $\mathcal{G}(u_{n})\to\mathcal{G}(u^{\dagger})$ in probability as $n\to\infty$. Therefore there exists a subsequence which satisfies (after labelling by $n$ again)

[TABLE]

Now let $\{\psi_{\ell}\}_{\ell=1}^{\infty}$ be an $r$-regular orthonormal wavelet basis for $L^{2}({\mathbb{R}}^{d})$ with $r\geq s$. We then, by (42), have

[TABLE]

For $\ell=1$, the above bound implies the existence of $\{u_{n_{1}(k)}\}_{k\in{\mathbb{N}}}\subset\{u_{n}\}_{n\in{\mathbb{N}}}$ and $\eta_{1}\in{\mathbb{R}}$ such that $\mathbb{E}|\langle u_{n_{1}(k)},\psi_{1}\rangle|\to\eta_{1}$ as $k\to\infty$. Considering $\ell=2,3,\dots$ successively one can similarly show the existence of $\{u_{n_{1}(k)}\}_{k\in{\mathbb{N}}}\supset\{u_{n_{2}(k)}\}_{k\in{\mathbb{N}}}\supset\dots\{u_{n_{\ell}(k)}\}_{k\in{\mathbb{N}}}\supset\dots$ and $\{\eta_{\ell}\}\in{\mathbb{R}}^{\infty}$ such that $\mathbb{E}|\langle u_{n_{\ell}(k)},\psi_{\ell}\rangle|\to\eta_{\ell}$ for any $\ell\in{\mathbb{N}}$ as $k\to\infty$. The subsequence $\{u_{n_{k}(k)}\}_{k\in{\mathbb{N}}}$ hence satisfies

[TABLE]

We relabel the above subsequence by $\{u_{n}\}_{n\in{\mathbb{N}}}$. Let $u^{*}:=\sum_{\ell=1}^{\infty}\eta_{\ell}\psi_{\ell}$. We have, by Jensen’s inequality,

[TABLE]

Hence $u^{*}\in B^{s}_{1}$. Let us now consider the strong convergence of the distribution in a larger space $B^{{\tilde{s}}}_{1}$ for $\tilde{s}<s$. Take $v\in B^{-{\tilde{s}}}_{\infty}(\mathbb{T}^{d})$ and write

[TABLE]

since $\mathbb{E}\|u_{n}\|_{B^{s}_{1}}$ is uniformly bounded, and as $\sum_{\ell=1}^{\infty}\ell^{\frac{{\tilde{s}}}{d}-\frac{1}{2}}\mathbb{E}|\langle u_{n}-u^{*},\psi_{\ell}\rangle|<\infty$ we can pass the expectation inside [47, theorem 1.38]. Noting that $\|u^{*}\|_{B^{s}_{1}}+{\mathbb{E}}\|u_{n}\|_{B^{s}_{1}}\leq C<\infty$, given any $\epsilon>0$, $N$ can be chosen large enough, independently of $n$, such that the second term in the last line of the above inequality is bounded by $\epsilon/2$. Having convergence coefficient-wise, there is $M\in{\mathbb{N}}$ large enough so that for $n>M$ the first term is bounded by $\epsilon/2$ as well. We therefore conclude that $u_{n}$ converges strongly to $u^{*}$ in $B^{\tilde{s}}_{1}(\mathbb{T}^{d})$ in probability. This then implies ([28, Lemma 4.2]) the existence of a subsequence, which we label $\{u_{n}\}_{n=1}^{\infty}$ again, such that

[TABLE]

This is true in particular for $t<s-d$. By continuity of $\mathcal{G}$ in $B^{t}_{1}(\mathbb{T}^{d})$, we conclude that

[TABLE]

This together with (43) give $\mathcal{G}(u^{\dagger})=\mathcal{G}(u^{*})$. ∎

Proof of corollary 4.8.

Since $B^{s}_{1}(\mathbb{T}^{d})$ is dense in $B^{t}_{1}(\mathbb{T}^{d})$, for any $\epsilon>0$, there exists $v\in B^{s}_{1}(\mathbb{T}^{d})$ such that $\|u^{\dagger}-v\|_{X}\leq\epsilon$. As $u_{n}$ is a minimizer of $I_{n}$ we have

[TABLE]

Rearranging and using Young’s inequality, similar to the previous proof, then gives

[TABLE]

Noting that $\|v\|_{B^{s}_{1}}\leq\|u^{\dagger}\|_{B^{s}_{1}}+\epsilon$ and |\Sigma^{-\frac{1}{2}}(\mathcal{G}(u^{\dagger})-\mathcal{G}(v))\big{|}\leq C\epsilon by local Lipschitz continuity of $\mathcal{G}$, we have

[TABLE]

This implies that

[TABLE]

Since $\epsilon$ was arbitrary, we conclude that \lim_{n\to\infty}\mathbb{E}|\Sigma^{-\frac{1}{2}}(\mathcal{G}(u^{\dagger})-\mathcal{G}(u_{n}))\big{|}^{2}=0, and therefore, |\Sigma^{-\frac{1}{2}}(\mathcal{G}(u^{\dagger})-\mathcal{G}(u_{n}))\big{|}\to 0 in probability. Hence, there exists a subsequence of $\{\mathcal{G}(u_{n})\}$ which converges to ${\mathcal{G}(u^{\dagger})}$ almost surely, giving the result. ∎

6.4 Proofs of results in section 5

Proof of theorem 5.8.

The probability distribution $\pi_{X}$ on ${\mathbb{R}}$ has finite Fisher information since

[TABLE]

As pointed out above we have $\lambda_{\ell}(A)=\lambda_{1}(\alpha_{\ell}A)$, for any $A\in{\mathcal{B}}({\mathbb{R}})$, where $\alpha_{\ell}=\ell^{s/d-1/2}$. By proposition 5.7 we have

[TABLE]

and, consequently, $D(\lambda)=B^{s^{\prime}}_{2}(\mathbb{T}^{d})$ for $s^{\prime}=s-\frac{d}{2}$. The rest follows from theorem 5.6. ∎

Acknowledgements

MB acknowledges support by ERC via Grant EU FP 7 - ERC Consolidator Grant 615216 LifeInverse. The work by TH was supported by the Academy of Finland via project 275177. Both MB and TH were further supported by the German Science Exchange Foundation DAAD via Project 57162894, Bayesian Inverse Problems in Banach Space.