A priori parameter choice in Tikhonov regularization with oversmoothing penalty for non-linear ill-posed problems

TL;DR

This paper investigates a priori parameter choice strategies in Tikhonov regularization for non-linear ill-posed problems, focusing on cases where solutions lack finite smoothness penalties, and establishes optimal convergence rates.

Contribution

It extends previous work to non-linear problems, providing optimal convergence rates for a priori parameter choices even with oversmoothing penalties.

Findings

Established optimal convergence rates for non-linear problems

Analyzed cases with non-finite penalty solutions

Extended linear case results to non-linear settings

Abstract

We study Tikhonov regularization for certain classes of non-linear ill-posed operator equations in Hilbert space. Emphasis is on the case where the solution smoothness fails to have a finite penalty value, as in the preceding study 'Tikhonov regularization with oversmoothing penalty for non-linear ill-posed problems in Hilbert scales'. Inverse Problems 34(1), 2018, by the same authors. Optimal order convergence rates are established for the specific a priori parameter choice, as used for the corresponding linear equations.

Equations

$$F(x)=y,$$

$$\Vert x-x^{†}{\Vert }_X\le \phi (\Vert F(x)-F(x^{†}){\Vert }_Y)$$

$$\Vert y-y^{\delta }{\Vert }_Y\le \delta$$

$$\mathcal{D}:=\mathcal{D}(F)\cap \mathcal{D}(B)\ne \varnothing$$

$$T_{\alpha }^{\delta }(x):=\Vert F(x)-y^{\delta }{\Vert }_Y^2+\alpha \Vert B(x-\bar{x}){\Vert }_X^2\to \min ,\text{mbox}subjecttox\in \mathcal{D},$$

$$\Vert Bx{\Vert }_X\ge m\Vert x{\Vert }_X,\text{mbox}forallx\in \mathcal{D}(B).$$

$$c_a\Vert x-x^{†}{\Vert }_{-a}\le \Vert F(x)-F(x^{†}){\Vert }_Y\le C_a\Vert x-x^{†}{\Vert }_{-a}\text{mbox}forallx\in \mathcal{D}.$$

$$x^{†}-\bar{x}\in M_{p,E}:=\left\{x\in X_p,\Vert x{\Vert }_p:=\Vert B^{p}x{\Vert }_X\le E\right\}.$$

$${\alpha }_{\ast }={\alpha }_{\ast }(\delta ):={\delta }^{\frac{2(a+1)}{a+p}}.$$

$$\Vert x_{{\alpha }_{\ast }}^{\delta }-x^{†}{\Vert }_X=\mathcal{O}\left({\delta }^{\frac{p}{a+p}}\right)\text{mbox}as\delta \to 0.$$

$$\Vert x_{{\alpha }_{\ast }}^{\delta }-x^{†}{\Vert }_{-a}\le K\delta$$

$$\Vert x_{{\alpha }_{\ast }}^{\delta }-x^{†}{\Vert }_p\le \tilde{E}$$

$$\Vert x_{{\alpha }_{\ast }}^{\delta }-x^{†}{\Vert }_X\le \Vert x_{{\alpha }_{\ast }}^{\delta }-x^{†}{\Vert }_{-a}^{\frac{p}{a+p}}\Vert x_{{\alpha }_{\ast }}^{\delta }-x^{†}{\Vert }_p^{\frac{a}{a+p}}.$$

$$T_{-a,\alpha }(x):=\Vert x-x^{†}{\Vert }_{-a}^2+\alpha \Vert B(x-\bar{x}){\Vert }_X^2,$$

$$\Vert x_{{\alpha }_{\ast }}-x^{†}{\Vert }_X$$

$$\Vert B^{-a}(x_{{\alpha }_{\ast }}-x^{†}){\Vert }_X$$

$$\Vert B(x_{{\alpha }_{\ast }}-\bar{x}){\Vert }_X$$

$$T_{-a,\alpha }(x_{{\alpha }_{\ast }})$$

$$\Vert x_{{\alpha }_{\ast }}-\bar{x}{\Vert }_p\le E,\text{and}\Vert x_{{\alpha }_{\ast }}-x^{†}{\Vert }_p\le E.$$

$$T_{{\alpha }_{\ast }}^{\delta }(x_{{\alpha }_{\ast }}^{\delta })\le T_{{\alpha }_{\ast }}^{\delta }(x_{{\alpha }_{\ast }}).$$

$$\Vert F(x_{{\alpha }_{\ast }}^{\delta })-y^{\delta }{\Vert }_Y\le C\delta ,$$

$$\Vert B(x_{{\alpha }_{\ast }}^{\delta }-\bar{x}){\Vert }_X\le C\frac{\delta }{\sqrt{{\alpha }_{\ast }}},$$

$$T_{{\alpha }_{\ast }}^{\delta }(x_{{\alpha }_{\ast }})$$

$$\le {\left(C_a\Vert x_{{\alpha }_{\ast }}-x^{†}{\Vert }_{-a}+\delta \right)}^2+E^2{\alpha }_{\ast }{\delta }^{2(p-1)/(a+p)}$$

$$\le {\left(C_{a}E\delta +\delta \right)}^2+E^2{\delta }^2$$

$$=\left((C_{a}E+1{)}^2+E^2\right){\delta }^2.$$

$$\Vert x_{{\alpha }_{\ast }}^{\delta }-x^{†}{\Vert }_{-a}$$

$$\le \frac{1}{c_a}\left(\Vert F(x_{{\alpha }_{\ast }}^{\delta })-y^{\delta }{\Vert }_Y+\Vert F(x^{†})-y^{\delta }{\Vert }_Y\right)$$

$$\le \frac{1}{c_a}\left(C\delta +\delta \right)=\frac{1}{c_a}\left(C+1\right)\delta =K\delta ,$$

$$\Vert B(x_{{\alpha }_{\ast }}^{\delta }-x_{{\alpha }_{\ast }}){\Vert }_X\le \tilde{C}\frac{\delta }{\sqrt{{\alpha }_{\ast }}}.$$

$$\Vert B(x_{{\alpha }_{\ast }}^{\delta }-x_{{\alpha }_{\ast }}){\Vert }_X\le \Vert B(x_{{\alpha }_{\ast }}-\bar{x}){\Vert }_X+\Vert B(x_{{\alpha }_{\ast }}^{\delta }-\bar{x}){\Vert }_X$$

$$\Vert x_{{\alpha }_{\ast }}^{\delta }-x_{{\alpha }_{\ast }}{\Vert }_p\le \bar{E}.$$

$$\Vert x_{{\alpha }_{\ast }}^{\delta }-x_{{\alpha }_{\ast }}{\Vert }_p\le \Vert x_{{\alpha }_{\ast }}^{\delta }-x_{{\alpha }_{\ast }}{\Vert }_1^{\frac{a+p}{a+1}}\Vert x_{{\alpha }_{\ast }}^{\delta }-x_{{\alpha }_{\ast }}{\Vert }_{-a}^{\frac{1-p}{a+1}}.$$

$$\Vert x_{{\alpha }_{\ast }}^{\delta }-x_{{\alpha }_{\ast }}{\Vert }_{-a}\le \Vert x_{{\alpha }_{\ast }}^{\delta }-x^{†}{\Vert }_{-a}+\Vert x_{{\alpha }_{\ast }}-x^{†}{\Vert }_{-a}.$$

$$\Vert x_{{\alpha }_{\ast }}^{\delta }-x_{{\alpha }_{\ast }}{\Vert }_p\le \bar{E}{\left(\frac{\delta }{\sqrt{{\alpha }_{\ast }}}\right)}^{{}^{\frac{a+p}{a+1}}}{\delta }^{\frac{1-p}{a+1}}=\bar{E}\delta {\alpha }_{\ast }^{-\frac{a+p}{2(a+1)}}=\bar{E}.$$

Full text

1. Introduction

The present paper is closely related to the recent work [4] of the authors published in the journal Inverse Problems devoted to the Tikhonov regularization for non-linear operator equation with oversmoothing penalties in Hilbert scales. Here we adopt the model and terminology. Since ibidem convergence rates of optimal order were only proven for the discrepancy principle as an a posteriori choice of the regularization parameter, we try here to close a gap in regularization theory by extending the same rate results to the case of appropriate a priori choices. This is in good coincidence with the corresponding results for linear operator equations presented in the seminal paper [8], where the same a priori parameter choice successfully allowed for order optimal convergence rates also in the case of oversmoothing penalties.

We consider the approximate solution of an operator equation

[TABLE]

which models an inverse problem with an (in general) non-linear forward operator $F:\mathcal{D}(F)\subseteq X\to Y$ mapping between the infinite dimensional Hilbert spaces $X$ and $Y$, with domain $\mathcal{D}(F)$. By $x^{\dagger}$ we denote a solution to (1) for given right-hand side $y$. As a consequence of the ‘smoothing’ property of $F$, which is typical for inverse problems, the non-linear equation (1) is locally ill-posed at the solution point $x^{\dagger}\in\mathcal{D}(F)$ (cf. [5, Def. 2]), which in particular means that stability estimates of the form

[TABLE]

cannot hold for all $x\in\mathcal{D}(F)$ in an arbitrarily small ball around $x^{\dagger}$ and for strictly increasing continuous functions $\varphi:[0,\infty)\to[0,\infty)$ with $\varphi(0)=0$. However, inequalities similar to (2), called conditional stability estimates, can hold on the one hand if the admissible range of $x\in\mathcal{D}(F)$ is restricted to densely defined subspaces of $X$. In this context, we refer to the seminal paper [2] as well as to [1, 6, 7] and references therein. On the other hand, they can hold for all $x\in\mathcal{D}(F)$ if the term $\|x-x^{\dagger}\|_{X}$ on the left-hand side of the inequality (2) is replaced with a weaker distance measure, for example a weaker norm (cf. [3] and references therein). In this paper, we follow a combination of both approaches in a Hilbert scale setting.

Based on noisy data $y^{\delta}\in Y$, obeying the deterministic noise model

[TABLE]

with noise level $\delta>0$, we use within the domain

[TABLE]

minimizers $x_{\alpha}^{\delta}\in\mathcal{D}$ of the Tikhonov functional $T^{\delta}_{\alpha}$ solving the extremal problem

[TABLE]

as stable approximate solutions (regularized solutions) to $x^{\dagger}$. Above, the element $\bar{x}\in\mathcal{D}$ is a given smooth reference element, and $B\colon\mathcal{D}(B)\subset X\to X$ is a densely defined, unbounded, linear self-adjoint operator, which is strictly positive such that we have for some $m>0$

[TABLE]

This operator $B$ generates a Hilbert scale $\{X_{\tau}\}_{\tau\in\mathbb{R}}$ with $X_{0}=X$, $X_{\tau}=\mathcal{D}(B^{\tau})$, and with corresponding norms $\|x\|_{\tau}:=\|B^{\tau}x\|_{X}$.

Here we shall focus on the case of an oversmoothing penalty, which means that $x^{\dagger}\not\in\mathcal{D}(B)$ such that $T^{\delta}_{\alpha}(x^{\dagger})=\infty$. In this case, the regularizing property $T^{\delta}_{\alpha}(x_{\alpha}^{\delta})\leq T^{\delta}_{\alpha}(x^{\dagger})$ does not provide additional value here. Throughout this paper, we assume the operator $F$ to be sequentially weakly continuous and its domain $\mathcal{D}(F)$ to be a convex and closed subset of $X$, which makes the general results of [9, Section 4.1.1] on existence and stability of the regularized solutions $x_{\alpha}^{\delta}$ applicable.

The paper is organized as follows: In Section 2 we formulate non-linearity and smoothness assumptions, which are required for obtaining a convergence rate result for Tikhonov regularization in Hilbert scales in the case of oversmoothing penalties under an appropriate a priori choice of the regularization parameter. Also in Section 2 we formulate the main theorem. Its proof will then follow from two propositions which are stated. Section 3 is devoted to proving both propositions. Section 4 completes the paper with some concluding discussions.

2. Assumptions and main result

In accordance with the previous study [4] we make the following additional assumption on the structure of non-linearity for the forward operator $F$ with respect to the Hilbert scale generated by the operator $B$. Sufficient conditions and examples for this non-linearity assumption can be found in the appendix of [4].

Assumption 1** (Non-linearity structure).**

There is a number $a>0$, and there are constants $0<c_{a}\leq C_{a}<\infty$ such that

[TABLE]

The left-hand inequality of condition (6) implies that, for the right-hand side $y$, there is no solution to (1) which belongs to $\mathcal{D}$. Moreover note that the parameter $a>0$ in Assumption 1 can be interpreted as degree of ill-posedness of the mapping $F$ at $x^{\dagger}$.

The solution smoothness is measured with respect to the generator $B$ of the Hilbert scale as follows. We fix the reference element $\bar{x}\in\mathcal{D}$, occurring in the penalty functional of $T^{\delta}_{\alpha}$.

Assumption 2** (Solution smoothness).**

There are $0<p<1$ and $E<\infty$ such that $x^{\dagger}\in\mathcal{D}(B^{p})$ and

[TABLE]

Moreover, we assume that $x^{\dagger}$ is an interior point of $\mathcal{D}(F)$, but $x^{\dagger}\notin\mathcal{D}(B)$.

We shall analyze the error behavior of the minimizer $x_{\alpha}^{\delta}$ to the Tikhonov functional $T^{\delta}_{\alpha}$ for the following specific a priori parameter choice.

Assumption 3** (A priori parameter choice).**

Given noise level $\delta>0$, ill-posedness degree $a>0$ (cf. Assumption 1) and solution smoothness $p\in(0,1)$ (cf. Assumption 2), let

[TABLE]

We shall occasionally use the identity $\frac{\delta}{\sqrt{\alpha_{\ast}}}=\delta^{\frac{p-1}{a+p}}$, and we highlight that for this parameter choice we have $\frac{\delta}{\sqrt{\alpha_{\ast}(\delta)}}\to\infty$ as $\delta\to 0$, for $0<p<1$.

The main result is as follows.

Theorem 1**.**

Under the assumptions stated above let $x_{\alpha_{\ast}}^{\delta}$ be the minimizer of the Tikhonov functional $T^{\delta}_{\alpha_{*}}$ for the a priori choice $\alpha_{*}$ from (8). Then we have the convergence rate

[TABLE]

This asymptotics is an immediate consequence of the following two propositions, the proofs of which will be given in the next section.

Proposition 1**.**

Under the a priori choice $\alpha_{*}$ from (8) and for sufficiently small $\delta>0$ we have that

[TABLE]

holds for some positive constant $K$.

Proposition 2**.**

Under the a priori choice $\alpha_{*}$ from (8) and for sufficiently small $\delta>0$ we have that

[TABLE]

holds for some positive constant $\tilde{E}$.

Proof of Theorem 1. Taking into account the assertions of Propositions 1 and 2, the convergence rate (9) follows directly from the interpolation inequality of the Hilbert scale $\{X_{\tau}\}_{\tau\in\mathbb{R}}$, applied here in the form

[TABLE]

Thus, the proof of the theorem is complete. ∎

3. Proofs of Propositions 1

and 2

Propositions 1 and 2 yield bounds in the (weak) $(-a)$-norm and in the (strong) $(p)$-norm, respectively. For the proofs we shall use auxiliary elements $x_{\alpha_{\ast}}$, constructed as follows. Precisely, in conjunction with the Tikhonov functional $T_{\alpha}^{\delta}$ from (4) we consider the artificial Tikhonov functional

[TABLE]

which is well-defined for all $x\in X$. Let $x_{\alpha}$ be the minimizers of $T_{-a,\alpha}$ over all $x\in X$ that are for all $\alpha>0$ independent of the noise level $\delta>0$ and recall now the parameter choice (8) from Assumption 3. For this choice of the regularization parameter the estimates from [4, Prop. 2] yield immediately the following assertions.

Proposition 3**.**

Suppose that $x^{\dagger}\in M_{p,E}$ for some $0<p<1$, and let $x_{\alpha}$ be the minimizer of $T_{-a,\alpha}$. Given $\alpha_{\ast}>0$ as in Assumption 3 the resulting element $x_{\alpha_{\ast}}$ obeys the bounds

[TABLE]

Moreover, we have that

[TABLE]

Notice, that in contrast to the solution element $x^{\dagger}$ the auxiliary element $x_{\alpha_{\ast}}$ belongs to $\mathcal{D}$, provided that $\delta$ is small enough, and hence we can use the minimizing property

[TABLE]

We derive the following consequence of Proposition 3 formulated in Proposition 4.

Proposition 4**.**

Let $x_{\alpha_{\ast}}^{\delta}$ be the minimizer of $T^{\delta}_{\alpha_{\ast}}$ for the Tikhonov functional $T_{\alpha}^{\delta}$ from (4) with the choice $\alpha_{\ast}$, as in Assumption 3, of the regularization parameter $\alpha>0$. Then we have for sufficiently small $\delta>0$ that

[TABLE]

where $C:=\left((C_{a}E+1)^{2}+E^{2}\right)^{1/2}$.

Proof.

Using (19) it is enough to bound $T^{\delta}_{\alpha_{\ast}}(x_{\alpha_{\ast}})$ by $C\delta$. We first argue that $x_{\alpha_{\ast}}$, the minimizer of the auxiliary functional (13) for $\alpha=\alpha_{*}$, belongs to the set $\mathcal{D}(F)\cap\mathcal{D}(B)$. Indeed, the bound (16) shows that $x_{\alpha_{\ast}}\in\mathcal{D}(B)$, and the bound (14) indicates that $x_{\alpha_{\ast}}\in\mathcal{D}(F)$ for sufficiently small $\delta>0$, where we use that $x^{\dagger}$ is an interior point of $\mathcal{D}(F)$. Now we find from (15) & (16), and from the right-hand inequality of (6), that

[TABLE]

This completes the proof of Proposition 4. ∎∎

Now we turn to the proofs of Propositions 1 and 2.

Proof of Proposition 1. Here we use the left-hand side in the non-linearity condition from Assumption 1 and find for sufficiently small $\delta>0$

[TABLE]

where $C$ is the constant from Proposition 4 and $K:=\frac{1}{c_{a}}\left(C+1\right)$. The proof is complete. ∎

In order to establish the bound occurring in Proposition 2 we start with the following estimate.

Lemma 1**.**

Let $\alpha_{\ast}$ be as in Assumption 3. Then there is a constant $\tilde{C}$ such that we have for sufficiently small $\delta>0$ that

[TABLE]

Proof.

By using the triangle inequality we find that

[TABLE]

The first summand on the right was bounded in (16), and the second was bounded in Proposition 4 for sufficiently small $\delta>0$. This yields the assertion with $\tilde{C}:=C+E$, where $C$ is the constant from Proposition 4. ∎∎

This allows for the following result.

Proposition 5**.**

There is a constant $\bar{E}$ such that we have for $\alpha_{\ast}$ as in Assumption 3 and for sufficiently small $\delta>0$ that

[TABLE]

Proof.

Again, we use the interpolation inequality, now in the form of

[TABLE]

The norm in the first factor was bounded in Lemma 1. The norm in the second factor can be bounded from

[TABLE]

Now we discuss both summands in the right-hand side of the inequality (22). The first summand was bounded in Proposition 1 by a multiple of $\delta$, and the same holds true for the second summand by virtue of (15). Therefore, there is a constant $\bar{E}$ such that

[TABLE]

This completes the proof of Proposition 5. ∎

We have gathered all auxiliary estimates in order to turn to the final proof.

Proof of Proposition 2. The estimate (11) is an immediate consequence of Proposition 5 and of the second bound from (18), overall yielding the constant $\tilde{E}:=\bar{E}+E$. This completes the proof.∎

4. Conclusions

We have shown that under the non-linearity Assumption 1 and the solution smoothness given as in Assumption 2 the a priori regularization parameter choice

[TABLE]

allows for the order optimal convergence rate (9) for all $0<p<1$. The obtained rate from Theorem 1 is valid for $0<p\leq a+1$ when using the same a priori choice of the regularization parameter from Assumption 3. In all cases, we have that $\alpha_{\ast}(\delta)\to 0$ as $\delta\to 0$. In the regular case with $1<p\leq a+1$ we also find the usual convergence $\frac{\delta^{2}}{\alpha(\delta)}\to 0$ as $\delta\to 0$. For the borderline case $p=1$ the quotient $\frac{\delta^{2}}{\alpha_{\ast}(\delta)}$ is constant. However, in the oversmoothing case $0<p<1$, as considered here, we find that $\frac{\delta^{2}}{\alpha_{\ast}(\delta)}\to\infty$ as $\delta\to 0$.

We stress another observation. In the regular case with $p>1$ we have the convergence property

[TABLE]

which is a consequence of the sequential weak compactness of a ball and of the Kadec-Klee property in the Hilbert space $X$. This cannot be shown for $0<p<1$ without using additional smoothness of the solution $x^{\dagger}$. Hence, the convergence property (23) as an implication of Theorem 1 is essentially based on the existence of a positive value $p$ in (7) expressing solution smoothness.

Acknowledgment

The research was financially supported by Deutsche Forschungsgemeinschaft (DFG-grant HO 1454/12-1) and by the Weierstrass Institute for Applied Analysis and Stochastics, Berlin.