Existence Theory for the EED Inpainting Problem
Michael Bildhauer, Marcelo C\'ardenas, Martin Fuchs, Joachim Weickert

TL;DR
This paper develops an existence theory for the edge-enhancing diffusion inpainting problem, proving the existence of solutions and their convergence properties in a nonlinear anisotropic diffusion framework.
Contribution
It introduces a rigorous mathematical existence framework for EED inpainting, applying fixed point theorems and analyzing solution sequences.
Findings
Existence of weak solutions established using Leray-Schauder theorem
Boundedness of all possible weak solutions shown
Convergence of iterative solution sequences demonstrated
Abstract
We establish an existence theory for an elliptic boundary value problem in image analysis known as edge-enhancing diffusion (EED) inpainting. The EED inpainting problem aims at restoring missing data in an image as steady state of a nonlinear anisotropic diffusion process where the known data provide Dirichlet boundary conditions. We prove existence of a weak solution by applying the Leray-Schauder Fixed point theorem and show that the set of all possible weak solutions is bounded. Moreover, we demonstrate that under certain conditions, the sequences resulting from iterative application of the operator from the existence theory contain convergent subsequences.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Existence Theory for the EED Inpainting Problem
Michael Bildhauer
Marcelo Cárdenas
Martin Fuchs
Joachim Weickert
Abstract
We establish an existence theory for an elliptic boundary value problem in image analysis known as edge-enhancing diffusion (EED) inpainting. The EED inpainting problem aims at restoring missing data in an image as steady state of a nonlinear anisotropic diffusion process where the known data provide Dirichlet boundary conditions. We prove the existence of a weak solution by applying the Leray-Schauder fixed point theorem and show that the set of all possible weak solutions is bounded. Moreover, we demonstrate that under certain conditions the sequences resulting from iterative application of the operator from the existence theory contain convergent subsequences.
Dedicated to Professor Nina Uraltseva on the occasion of her 85th birthday.
AMS Subject Classification: 35J57, 94A08
Keywords: boundary value problems, anisotropic diffusion, Leray-Schauder fixed point theorem, inpainting, image restoration, image compression
1 Introduction
Restoring missing data in an EED inpainting problem means that we are given an open set as the area of an image and that the image data are just partially known, i.e. we consider a subset and a function : . Here the values of represent the grey level between pure white and black.
As a typical example we may imagine that is a finite union of tiny regions like it is illustrated in Figure 1 below.
The complete image is recovered by filling in the missing information at .
To this purpose, the boundary value problem
[TABLE]
is studied, where denotes the outward unit normal to and where for any given function a Gaussian-smoothed version of with some fixed parameter is denoted by .
In fact, this problem serves as a mathematical model for the anisotropic diffusion image inpainting problem [11, 13, 4].
The Dirichlet boundary condition (2) enforces the inpainted result to be coherent with the known data at the boundary of the known data region . The problem is also supplemented with the natural Neumann boundary condition (3).
The diffusion tensor : depending on the gradient of is usually designed in order to steer the diffusion process in such a way that important geometrical information is taken into account.
Before going into details, we have to clarify our notation and assumptions regarding problem (1)–(3).
Notation.
In the following we always suppose to be a bounded Lipschitz domain. For the definition of the Lebesgue- and Sobolev spaces , and its variants we refer to the monograph [1].
If is a given positive parameter, then throughout this paper we denote for any function the globally smoothed function with the symbol ,
[TABLE]
where denotes the Gaussian kernel
[TABLE]
We use the symbol to denote a generic constant which may change from line to line. Whenever we want to stress its dependence on other values , we write
Main assumptions.
- i)
We consider a finite number of open disjoint Lipschitz domains , , , …, , s.t. for any , . We then define
[TABLE]
For the sake of simplicity we assume throughout this paper that , hence . Otherwise we would have to decompose with
[TABLE] 2. ii)
We suppose that the function : is of class . Extending this function to a function of class , we use the same symbol with a slight abuse of notation. 3. iii)
The diffusion tensor is assumed to be a continuous function satisfying the symmetry condition
[TABLE]
Moreover, our hypothesis on non-degenerate ellipticity reads as: there exist positive constants , , such that
[TABLE]
EED inpainting.
The choice of the ellipticity condition (6) covers the inpainting problem with Edge-Enhancing Diffusion (EED) [11, 13, 4].
As an easy example, may look like (in fact its representation along the edge direction and across the edge direction, respectively, and the notion ”edge direction” is motivated by the boundary of level sets of )
[TABLE]
In other words, the diffusion tensor is such that the differential operator encourages diffusion along edges over diffusion across edges.
More precisely, is defined as a matrix with eigenvectors and having as corresponding eigenvalues and where is the Charbonnier diffusivity [3] given by
Note that the inpainted image crucially depends on the steering parameter . Concerning this, a rigorous analysis remains an open problem.
However, for the rest of this work we may assume w.l.o.g. that except when stated explicitly otherwise.
Edge-enhancing anisotropic diffusion was first proposed as a parabolic evolution process for denoising [11], where it can be regarded as an anisotropic alternative to isotropic regularisations [2] of the Perona-Malik filter [9]. This evolution has been shown to be well-posed in the continuous, space discrete, and fully discrete setting [12]. Later on, the elliptic steady state equation of EED has been supplemented with Dirichlet data and used for inpainting missing regions in matrix-valued images [13].
Its main application today is inpainting-based lossy image compression, where only a sparse, carefully optimised subset of the data is stored and the missing data are recovered by EED inpainting [4]. In this context, experiments have shown that EED gives state-of-the-art results that outperform other partial differential equations in terms of reconstruction quality [10].
Fig. 1 depicts an example for EED inpainting of sparse data, using a photo of Professor Nina Uraltseva. In spite of its qualities in practical applications, there is no existence theory for EED inpainting so far. Our paper closes this gap.
Main goal and summary.
In the present work, we show the existence of a weak solution to problem (1)–(3) and present some properties of the set of possibly multiple weak solutions. Moreover, the canonical iteration procedure is analyzed by the way proving -bounds in different settings.
Our paper is organized as follows: Section 2 covers the existence of a solution to (1)–(3). We define an operator : via an associated variational problem and the Leray-Schauder fixed point theorem [5, 6] gives a fixed point of and in turn the desired weak solution.
The analysis of the fixed point set outlined in Section 3 is motivated by the fact, that no results on the uniqueness of fixed points are available. Here we characterize the fixed points by another variational formulation.
The last section follows the idea, that suitable iterations of the operator may serve as a good approximation for a fixed point of .
For this kind of iteration, we shortly establish -bounds in the case of sufficiently large which in general fail to be true. Thus, uniform a priori estimates are based on a suitable smallness assumption.
Given these -bounds, we are able to show the existence of fixed points w.r.t. an iterated operator . We emphasize that these fixed points are found in a ball whose diameter just depends on the data and is independent of .
Concluding remarks deal with an interpretation of our error estimates.
2 Existence theory
The goal of this section is to show that under the above conditions problem (1)–(3) has at least one weak solution. Our strategy is to apply the Leray-Schauder fixed point theorem:
First we define the weak solutions of the problem as the fixed points of an appropriate operator and then show that the operator has at least one fixed point [5, 6]. Note that is fixed throughout this section.
Let us begin by introducing the relevant operator. For this purpose we need the following two ingredients: the class
[TABLE]
and the family of functionals : defined as
[TABLE]
for any and any
Definition 2.1**.**
The operator is defined for any as the argumentum minimi (w.r.t. the class )
[TABLE]
For the sake of notational simplicity, the dependence on is not highlighted unless we need a careful analysis proving uniform bounds in the last section.
This definition is well posed on account of Proposition 2.2, whose proof follows from the ellipticity of the diffusion tensor in a well known manner. We just recall the basic ingredients concerning Sobolev functions which in addition can be used as precise references throughout our whole exposition. The continuity of the trace operator follows, e.g., from Theorem 3.4.5 of [8], the variant of Poincaré’s inequality from Theorem 3.6.4 of [8]
Proposition 2.1**.**
Let be a bounded open Lipschitz domain.
- i)
(Continuity of the trace operator)* *
For any there exists a bounded linear operator (the trace operator) such that whenever In particular, there exists a constant such that
[TABLE]
for any 2. ii)
(Poincaré inequality)* *
Let with . If with is such that in the trace sense, then
[TABLE]
for a positive constant which does not depend on
Proposition 2.2**.**
The set is a convex and weakly closed subset of Moreover, for any fixed the functional is weakly lower semicontinuous over and has a unique minimiser in . The unique minimiser satisfies for all
[TABLE]
and for all such that the Euler-Lagrange equation
[TABLE]
Weak solution.
Any fixed point of the operator is a weak solution of (1)–(3) in the following sense:
- i)
By definition, any fixed point of satisfies the Euler-Lagrange equation (12), hence is a weak solution of (1) in . 2. ii)
Any fixed point is of class , thus (2) holds. 3. iii)
Suppose that and that there exists an open neighborhood s.t. is globally of class on . Then we have (3) a.e. on .
We are now ready to state our main result:
Theorem 2.1**.**
(Existence of a weak solution) Suppose that we have our general hypotheses as stated above. Then the operator has at least one fixed point, i.e. the boundary value problem (1)–(3) is weakly solvable.
The proof of Theorem 2.1 is divided into different steps which we present as intermediate lemmas.
Lemma 2.1**.**
Let then
[TABLE]
for some positive constant which is independent of
Proof. First we estimate in terms of . It holds for any that
[TABLE]
for , . With (14) we find s.t.
[TABLE]
and using the same argument, (15) is established for all higher order derivatives.
Next we discuss the size of Fixing , we have by Hölder’s inequality that
[TABLE]
where for the last inequality we applied the ellipticity condition (6). Moreover, choosing as comparison function in (11) and applying (6) once again, we obtain
[TABLE]
whereas on account of (15),
[TABLE]
Applying these estimates of and to (2), we obtain
[TABLE]
Since is in the class , thus coinciding with over we may apply the Poincaré inequality (10) to the difference and obtain
[TABLE]
Combining this estimate with inequality (17) we end up with (13) for a constant depending on the data , and but not depending on the function under consideration.
Lemma 2.2**.**
The mapping : is continuous.
Proof. Let and be such that
[TABLE]
Moreover, let and . We have to show that
[TABLE]
In order to prove (19), we begin with an observation concerning the mollifications: recalling (15) and (18) we have
[TABLE]
where we use the symbol instead of . The ellipticity condition (6), the continuity of the diffusion tensor and (20) imply for all and for all
[TABLE]
with constants , .
We again turn our attention to the sequence and combine the minimality of (recall (11), use as admissible comparison function) with inequality (21) to obtain
[TABLE]
Moreover, since we may apply Poincaré’s inequality (10) in order to get the estimate
[TABLE]
again with a constant just depending on the data. Inequalities (22) and (23) are put together with the following result of the first step:
[TABLE]
Then, by the Banach-Alaoglu theorem, there exists a subsequence of s.t. for some
[TABLE]
In the next step we show that in fact which immediately gives the convergence of the sequence as a whole:
[TABLE]
In fact, since is a closed convex subset of , Mazur’s lemma implies that it is also weakly closed, hence .
On the other hand, we have (12) w.r.t. the minimisers : it holds that
[TABLE]
for any satisfying . Hence, applying (20) and (25) in equation (27) we arrive at the limit (w.r.t. ) equation
[TABLE]
By definition is a solution of (28) as well which, by uniqueness, is just possible provided that and we have established our claim (26).
It remains to improve (26) towards
[TABLE]
which gives (19) as a byproduct. We have the Euler-Lagrange equations (12) both for and for with as admissible test function
[TABLE]
which, together with the ellipticity (6), shows for some
[TABLE]
We finally observe that uniformly on (by the continuity of and by (20)) which, together with the uniform bound (24), gives (29) and the proof is completed.
The last auxiliary lemma reads as:
Lemma 2.3**.**
The continuous mapping : is compact.
Proof. Suppose that we are given a sequence s.t.
[TABLE]
Letting we then have to extract a convergent subsequence ,
[TABLE]
where we do not claim that is a solution of some limit equation.
An account of our hypothesis (30) we can exactly follow the lines outlined in the proof of Lemma 2.2 and reproduce (24) in the situation at hand. Passing to a subsequence we may suppose
[TABLE]
We finally establsh -convergence of the gradients, which in particular proves the claim (31).
In fact, observe that on account of (15) and (30) the functions
[TABLE]
are bounded and equicontinuous on . Thus by Arzela’s theorem there exist a further subsequence, with a slight abuse of notation still denoted by , and a function : s.t. as
[TABLE]
We write
[TABLE]
The uniform convergence implies (recalling (24))
[TABLE]
As in the proof of Lemma 2.2, the Euler-Lagrange equation w.r.t. is applied leading to
[TABLE]
Here the second term is handled with the weak convergence of (see (32)), for the first one we argue in addition with the uniform convergence to obtain
[TABLE]
Summarizing the results, (2), (34) and (35) yield the main equality
[TABLE]
As a final remark we just mention that the condition
[TABLE]
for all , and for some positive constant is induced by the uniform convergence, which completes the proof.
Proof of Theorem 2.1.
The theorem is proved once we have verified the assumptions of [5], Theorem 11.3.
By Lemma 2.3, is compact mapping and it remains to prove the existence of a positive constant with the property
[TABLE]
Lemma 2.1 in particular shows
[TABLE]
W.l.o.g. we may assume . If , then
[TABLE]
and using Young’s inequality, we obtain for any
[TABLE]
By elementary calculations we may choose
[TABLE]
letting in addition and the claim follows.
3 Analysis of the fixed point set
In the previous section we proved the existence of at least one solution to (1)–(3) using the Leray-Schauder fixed point argument.
Since no information on the uniqueness of solutions is available, the analysis of the fixed point set is of particular interest. In this section we study the properties of the set of possibly multiple weak solutions.
We still fix a smoothing parameter throughout this section.
In Proposition 3.1 we show that the set
[TABLE]
of fixed points of is bounded.
Then, in Proposition 3.2, we introduce an appropriate functional and characterize the set as the set of all weak -limits of -minimizing sequences.
Proposition 3.1**.**
The set is a bounded subset of .
Proof. By definition we have (recall (7)). Thus, Poincaré’s inequality (10) can be applied to any function and we find a constant s.t.
[TABLE]
Lemma 2.1 yields for
[TABLE]
Youngs’ inequality then immediately gives
[TABLE]
Note that (37) implies as above
[TABLE]
Finally we may apply (15) once again together with the ellipticity condition (6), which proves the proposition on account of
[TABLE]
where the last inequality follows from the minimality of .
Next we consider the functional
[TABLE]
and the set of cluster points
[TABLE]
Proposition 3.2**.**
*The sets and coincide, i.e. . *
Proof. The inclusion follows (considering a fixed point as a contant sequence) from Proposition 3.1 which gives . Moreover, we observe that the existence of a fixed point (Theorem 2.1) implies
[TABLE]
Now fix any -minimizing sequence with weak -cluster point , i.e. we suppose after passing to a subsequence (not relabeled)
[TABLE]
and claim that is a fixed point of , i.e. . Note that is weakly closed, i.e. .
We have
[TABLE]
and by the -minimizing property of the sequence
[TABLE]
From (43) and (40) we a find subsequence s.t.
[TABLE]
where we also have . Moreover, by (15) and Arzela’s theorem, we may assume the uniform convergence on as :
[TABLE]
Arguing with lower semicontinuity (see (41)), the uniform convergence (42), the minimality of and once again with (42) we have for any
[TABLE]
This minimizing property of implies by the uniqueness of solutions . Finally pointwise a.e. convergence of and show the claim .
Remark 3.1**.**
In fact, any -minimizing sequence satisfies
[TABLE]
which guarantees the existence of weakly convergent subsequences.
Proof. Fix a -minimizing sequence . The inequality
[TABLE]
together with (recall (38))
[TABLE]
yields for any
[TABLE]
provided that is sufficiently large. Exactly as outlined in the proof of Proposition 3.1 we derive
[TABLE]
and with Young’s inequality we get
[TABLE]
On account of (43) we now may follow the concluding remarks proving Proposition 3.1 with the result
[TABLE]
hence
[TABLE]
Since the sequence is -minimizing, Remark 3.1 is obvious.
4 Study of iterated sequences
In this section we are interested in the question, whether the above considerations provide a rigorous analytical framework for an iteration of the operator as approximation of the EED inpainting problem under consideration.
More precisely, recalling (8) we fix s.t. on and let
[TABLE]
Here and in the following the dependence of the operator on the parameter is of particular interest. Nevertheless we keep the notation of the previous sections and do not highlight the dependence on .
In Section 4.1 it becomes evident, that for arbitrary values of we even cannot expect uniform -estimates for an iteration of . We just shortly sketch that assuming to be large enough leads to -bounds without further constraints on the data.
In Section 4.2 some refined a priori estimates are presented which lead to fixed points of the iterated operator in a priori bounded balls . However, these estimates rely on a suitable smallness condition on the data.
We finish this section with some concluding remark on error estimates.
4.1 Iterated sequences with large
Proposition 4.1**.**
Consider as given in (44). There exists a constant (compare (4.1)) s.t. in case
[TABLE]
Proof. For any we have , hence by the definition off the Gaussian kernel
[TABLE]
Let us recall Poincaré’s inequality (10) by writing ,
[TABLE]
Specifying the constants occuring in (2) and recalling the uniform ellipticity (6) we obtain
[TABLE]
Now let
[TABLE]
thus (45), (46) and (47) yield
[TABLE]
Applying iteratively (49) to we obtain that
[TABLE]
Hence, if we assume , then a final application of Poincaré’s inequality shows the proposition.
Using the compactness of the operator and the same reasoning as in Lemma 2.2 we immediately obtain as a byproduct
Corollary 4.1**.**
Consider as given in (44) and suppose that with given in (4.1)).
- i)
There exists a subsequence and a function such that as . 2. ii)
We have
[TABLE]
4.2 Iterated sequences with small
Throughout this section, is some arbitrary (small) fixed positive real number. In particular, is not bounded from below by the data of the problem.
4.2.1 A priori estimates
Here we derive the main tool for the analysis of the sequence , where we recall the definition (44) of this sequence and once more emphasize the ellipticity condition (6), where now is handled as a free parameter.
Similar to (2) one observes for , on ,
[TABLE]
Condition (6) and the minimality of give as before
[TABLE]
Using the elementary inequality
[TABLE]
we further obtain (now emphasizing the precise constants)
[TABLE]
Thus, it is shown that for all , on ,
[TABLE]
Note that, for instance, the constant occurring in (15) strongly depends on , thus we now proceed in a different manner using an integration by parts
[TABLE]
Next, the well known property
[TABLE]
is used. Denoting the constant in the trace inequality (9) by , the second term on the r.h.s. of (52) is handled via (recalling Poincaré’s inequality)
[TABLE]
Altogether, (52), (53) and (54) imply
[TABLE]
[TABLE]
Define the constants
[TABLE]
and reformulate (56) using (57) and (58) as
[TABLE]
which allows us to prove:
Theorem 4.1**.**
With the notation of above we have for all
[TABLE]
Proof by induction. For inequality (59) implies
[TABLE]
which is (60) for . Now suppose that (60) holds for some . As above for and for given , , …, the elementary estimate
[TABLE]
is used and again with the help of (59) we obtain
[TABLE]
Corollary 4.2**.**
*Let us suppose that in addition to the assumptions of Theorem 4.1 we have , where is the constant defined in (57).
Moreover, fix and suppose that is a subset of such that for all we have on and
[TABLE]
- i)
Then there is a uniform constant such that for all , v=v^{(n)}=T\big{(}v^{(n-1)}\big{)}, , we have
[TABLE] 2. ii)
For a universal constant not depending on we have i) for any .
Proof. Once again recalling (10), the proof follows from Theorem 4.1.
Remark 4.1**.**
As it should be expected, the smallness of corresponds to or small or small or large.
4.2.2 Iterated fixed points on a priori sets
In this subsection we derive the existence of iterated fixed points in a priori local bounded subsets of .
In order to find a suitable domain s.t. : , we first let for any fixed
[TABLE]
We then fix some , where for our purposes we may suppose in the following that on for any .
Now let
[TABLE]
Clearly, if then or for some and some , hence
[TABLE]
With the additional notation
[TABLE]
the same reasoning as above shows for all
[TABLE]
Increasing in Corollary 4.2, , if necessary, and choosing sufficiently large, we now choose
[TABLE]
such that for all
[TABLE]
Since is a closed convex subset of , the above equality shows the same for , and Corollary 11.2 of [5] provides for all at least one fixed point of in , hence in .
4.2.3 Error estimates
The numerical discussion of iterated fixed points is motivated by letting for all , and for a given as above (recall (44)
[TABLE]
With the notation
[TABLE]
we immediately obtain the first observation:
[TABLE]
i.e. provides a measure for the failure of to be numerically detected as one possible fixed point w.r.t. .
Of course it is possible to choose subsequences and such that as (recall Theorem 4.1)
[TABLE]
These simple observations lead to the analysis of the operator . In fact, if is a fixed point w.r.t. ,
[TABLE]
then we have
[TABLE]
Now a given fixed point of in general is not simultaneously a fixed point of although the converse trivially is true. The error evidently has to enter the left hand side of (61).
Note that we then apply the iterated operator to . This is crucial for the numerical interpretation: Decreasing in the kernel appearing in means to blow up the error via the non-contracting operator in which exactly describes the behavior of the examples sketched in the introduction.
Acknowledgement
The research of M.C. and J.W. has received funding by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement no. 741215, ERC Advanced Grant INCOVID).
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] Adams, R.A.: Sobolev Spaces. Academic Press, London (1975)
- 2[2] Catté, F., Lions, P.L., Morel, J.M., Coll, T.: Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis 32 , 1895–1909 (1992)
- 3[3] Charbonnier, P., Blanc-Féraud, L., Aubert, G., Barlaud, M.: Two deterministic half-quadratic regularization algorithms for computed imaging. In: Proc. 1994 IEEE International Conference on Image Processing. vol. 2, pp. 168–172. IEEE Computer Society Press, Austin, TX (Nov 1994)
- 4[4] Galić, I., Weickert, J., Welk, M., Bruhn, A., Belyaev, A., Seidel, H.P.: Image compression with anisotropic diffusion. Journal of Mathematical Imaging and Vision 31 (2–3), 255–269 (Jul 2008)
- 5[5] Gilbarg, D., Trudinger, N.S.: Elliptic Partial Differential Equations of Second Order. Springer, Berlin, revised third printing, second edn. (1998)
- 6[6] Ladyzhenskaya, O.A., Uraltseva, N.N.: Linear and Quasilinear Elliptic Equations. Academic Press, New York (1968)
- 7[7] Mainberger, M., Hoffmann, S., Weickert, J., Tang, C.H., Johannsen, D., Neumann, F., Doerr, B.: Optimising spatial and tonal data for homogeneous diffusion inpainting. In: Bruckstein, A.M., ter Haar Romeny, B., Bronstein, A.M., Bronstein, M.M. (eds.) Scale Space and Variational Methods in Computer Vision, Lecture Notes in Computer Science, vol. 6667, pp. 26–37. Springer, Berlin (2012)
- 8[8] Morrey, C.B.: Multiple Integrals in the Calculus of Variations. Springer, Berlin (1966)
