Double phase image restoration
Petteri Harjulehto, Peter H\"ast\"o

TL;DR
This paper investigates the use of double phase functionals for image restoration, focusing on mathematical properties and convergence of energy minimizers in the context of bounded variation functions.
Contribution
It introduces a novel analysis of double phase energy minimizers for BV functions and establishes their connection via $Gamma$-convergence and relaxation techniques.
Findings
Double phase energy minimizers are characterized for BV functions.
The energy can be obtained through $Gamma$-convergence of regularized functionals.
A capped fractional maximal function is used as a key analytical tool.
Abstract
In this paper we explore the potential of the double phase functional in an image processing context. To this end, we study minimizers of the double phase energy for functions with bounded variation and show that this energy can be obtained by -convergence or relaxation of regularized functionals. A central tool is a capped fractional maximal function of the derivative of functions.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Double phase image restoration
Petteri Harjulehto
Petteri Harjulehto, Department of Mathematics and Statistics, FI-20014 University of Turku, Finland
and
Peter Hästö
Peter Hästö, Department of Mathematics and Statistics, FI-20014 University of Turku, Finland, and Department of Mathematics, FI-90014 University of Oulu, Finland
Abstract.
In this paper we explore the potential of the double phase functional in an image processing context. To this end, we study minimizers of the double phase energy for functions with bounded variation and show that this energy can be obtained by -convergence or relaxation of regularized functionals. A central tool is a capped fractional maximal function of the derivative of functions.
Key words and phrases:
Image restoration, double phase, bounded variation, Gamma-convergence, relaxation, fractional maximal function
2010 Mathematics Subject Classification:
49J45; 49N45, 94A08
1. Introduction
The double phase functional was introduced in the 1980s by Zhikov [46], but has only recently become the focus of intense research, starting in 2015 with Baroni, Colombo and Mingione [5, 6, 15, 17]. Subsequently, many other researchers studied double phase problems as well, see, e.g., [9, 18, 20, 22, 39, 40] for regularity theory, [10, 21, 43] for Calderón–Zygmund estimates and [27, 36, 37] for some other topics. Generalizations of the double phase functional have been studies, e.g. in [7, 25, 26, 28, 33, 34, 38, 45].
Zhikov’s original motivation for his functionals with non-standard growth was modelling physical phenomena. Another of his models, the variable exponent functional, was later applied also to the context of image processing, see [1, 14, 31, 35]. In this article, we demonstrate the potential also of the double phase functional in the image processing domain. This is to the best of our knowledge the first paper to consider the double phase functional in the space of functions of bounded variation.
In mathematical image processing, we interpret a function as the gray-scale intensity at each location. If the function is discretized, we obtain an array of pixels common in computer implementations. Typically, is a rectangle and the image contains different objects whose edges correspond to discontinuities of . The presence of discontinuities makes this field challenging to approach with tools of analysis, but the space has proven useful. We refer to the book [4] by Aubert and Kornprobst for an overview of PDE-based image processing.
The classical ROF-model [41] for image restoration calls for minimizing the energy
[TABLE]
where is the given, corrupted input image that is to be restored. Here is a fidelity term which forces to be close to on average, whereas the regularizing term limits the variation of . This model is known to be prone to a stair-casing or banding effect whereby piecewise constant minimizers are often produced [13]. On the other hand, replacing by leads to a heat-equation type problem, and solutions which are . This is not usually desirable in the image processing context, as edges become blurred.
The energy of the double phase functional combines growth with two different powers. It is given by the expression
[TABLE]
Here is a bounded function and . All the previously mentioned double-phase references concern super-linear growth (usually , but see also [22]). However, for image processing, the case and is especially interesting (see above and the discussion in [14]). Then the first term corresponds to the ROF-model, whereas the second term introduces a smoothing effect when . The parameter is chosen such that at the edges in the image and elsewhere. Usually, the location of the edges is not known, so in applications is estimated from the initial data . Then this adaptive model can avoid the stair-casing effect of the ROF-model.
In the case , the double phase energy must naturally be studied in a space of -type. It is not difficult to prove existence of the minimizer even in this case (cf. Proposition 2.4). However, the -space is quite ill-behaved, so it is useful for practical implementations to approximate the energy by more regular functionals (see, e.g., [44, Section 6] in the image processing context). The notion of -convergence is often employed in this situation [8, 19], and this article is no exception: our main result (Theorems 4.1 and 4.2) shows that the double phase functional (with fidelity term)
[TABLE]
can be approximated in the sense of -convergence by both
[TABLE]
Finally, in Corollary 4.3, we show that the double phase functional can be understood as the relaxation of the double phase functional.
Note that we use inside the power-function, . This is of course equivalent to having another function outside, but it turns out that the condition on can be more conveniently expressed with this formulation (see Remark 3.3).
2. Notation and existence of minimizers of bounded variation
We consider subsets of the Euclidean space , . The most interesting case for image processing is , but we can include higher dimensions without extra complication. By we denote a bounded domain, i.e. an open and connected set. The notation means that there exists a constant such that . By we denote a generic constant whose value may change between appearances. Let be non-negative. By we denote the weighted Lebesgue space with weight , given by the norm
[TABLE]
is the corresponding Sobolev space. Note that we use the “weight as multiplier” formulation, so the corresponding weighted measure is , not . By we denote the -dimensional Hausdorff measure. By we denote the total variation measure of a vector measure , defined as
[TABLE]
By we denote the Hardy–Littlewood maximal function of .
A function has bounded variation, denoted , if
[TABLE]
Note that this quantity is sometimes denoted by . We follow the notation of [3], which is convenient since it turns out that is the total variation of a vector measure . Furthermore, can be decomposed as
[TABLE]
where is the absolutely continuous part of the derivative, is the essential point-wise jump of the function, is the normal of the level-set, is a set of Hausdorff dimension at most [2, Theorem 2.3] and the Cantor part has the property that if [3, Proposition 3.92]. The space has the following precompactness property [3, Proposition 3.13]: if \sup_{i}\big{(}|Du_{i}|(\Omega)+\|u_{i}\|_{L^{1}(\Omega)}\big{)}<\infty, then there exists a subsequence, denoted again by , and such that
[TABLE]
The derivative of the convolution of a -function can be calculated as expected using either the derivative-measure or the function [3, Proposition 3.2 and equation (2.2)]:
[TABLE]
We refer to [2, 3, 8] for more information about spaces.
We abbreviate and define for and initial data the double phase functional
[TABLE]
for measurable . We can easily show the existence of a minimizer for this functional using the direct method of calculus of variations:
Proposition 2.4**.**
There exists a unique minimizer , i.e.
[TABLE]
Proof.
Let be a minimizing sequence, that is with
[TABLE]
By -precompactness (2.2) there exists a subsequence, denoted again by , such that in and . The space is reflexive [29, Theorem 3.6.8], so we can find a weakly convergent subsequence . By [24, Theorem 2.2.8], the modular in is weakly lower semicontinuous, so that
[TABLE]
The inequality for the term follows analogously. Hence is a minimizer.
Finally, we note that the and parts are convex and the part is strictly convex, so the usual argument yields uniqueness, namely, if and are distinct minimizers, then we obtain a contradiction from . ∎
3. Lower estimates for the double phase functional
To be able to construct the minimizers of with some numerical scheme, we must show that the double phase functional can be approximated by some more regular variants. We regularize the functional by adding either to the exponent of the first term (so that the problem is in ) or to the weight (in which case the problem is in ). For brevity, we present the proof only for one case which includes both these regularizations:
[TABLE]
We start with a lower bound for , which is the more difficult part.
Lemma 3.1**.**
Let be closed and . For and , there exist in a neighborhood of such that
[TABLE]
Proof.
Let be the convolution with the standard mollifier and assume that . By [30, Lemma 4.5] and classical -results
[TABLE]
For the term with the weight , we consider two cases and use the different expressions from (2.3). If for all , then
[TABLE]
note that the condition with ensures that is absolutely continuous in and note also that the last inequality follows from elementary estimates (e.g. [24, Lemma 4.6.3]). Furthermore, since and the maximal operator is bounded on , we see that the function on the right-hand side is in , as well. If , then the estimate trivially holds. Suppose then that for some . Since , we obtain the inequality
[TABLE]
so that . Therefore
[TABLE]
where we used that for the middle step. Again, since , we obtain an upper bound independent of in the space . In the set we have almost everywhere. Thus it follows by dominated convergence in that
[TABLE]
We have so far shown that
[TABLE]
It remains to change the first functional from to . Equation (2.3) implies , where depends on . Therefore
[TABLE]
We choose so that and and set . Then
[TABLE]
Remark 3.3*.*
From the previous proof we can see that the exact condition used for is not , but rather the inequality
[TABLE]
This means that we could replace in the double phase functional with for as long as . This kind of condition was first identified for the double phase functional in [29, Section 7.2].
With the method of the previous proof, one can obtain from (3.2) that is bounded by when and denotes the fractional maximal operator (cf. Lemma 3.5). This will allow us to prove the result for bounded functions with a larger class of weights . A number of recent studies, e.g. [11, 12], deal with the question of the Sobolev regularity of the maximal function of a Sobolev or function . However, we have not found any results on the maximal function of the derivative of a function. Therefore, the following result may be of independent interest.
Proposition 3.4**.**
*Let be a vector Borel measure in with finite total variation , and . Then the capped fractional maximal function *
[TABLE]
belongs to if .
Furthermore, the bound is sharp since the claim does not hold for .
Proof.
We consider dyadic cubes intersecting with side-length at most . Specifically, we assume that the cubes are of the form and denote by the set of such cubes with side-length . Let be the cube which contains and be its threefold dilate. We define . If , then . Thus
[TABLE]
where and is the smallest integer with . We raise this to the power and estimate the supremum by a sum:
[TABLE]
Next we integrate over and use that can be estimated by the sum of terms of the form with . Thus we obtain that
[TABLE]
Let us maximize the sum separately for each . Since is a partition of , we can write this optimization problem as
[TABLE]
where for ; the last restriction holds since by the definition of . We consider what values of the ’s leads to a maximally large sum. If , then
[TABLE]
for . Therefore the sum is maximized subject to the constraints when for as many indices as possible and zero for the rest. There are no more than such maximal indices. Thus
[TABLE]
We use this estimate in our previous inequality, and conclude that
[TABLE]
The last sum is finite if , which is equivalent to the condition in the proposition.
It remains to prove sharpness. For simplicity we consider only the case when is an integer. We let be a -dimensional plane and define . Denote . Then
[TABLE]
We raise this to the power and integrate over :
[TABLE]
This integral diverges if , which gives the claimed bound for . In the case of non-integer , we instead choose our set as the Cartesian product of a plane and a Cantor set, and estimate as before. ∎
With the fractional maximal operator we can extend Lemma 3.1 in the case of bounded functions. Bounded functions are very natural in the context of image processing, since the grey-scale values are usually taken in some compact interval such as or . Note that to use the previous proposition, we cannot directly move to the total variation measure , since this is not in general going to satisfy the appropriate decay when is bounded. Rather, we have to first estimate the absolute value of the measure of a ball, , and only afterward move to . In the next result we therefore work with the vector measure rather than its total variation, which makes the estimates slightly more difficult.
Lemma 3.5**.**
Let be closed and for some . For and , there exist in a neighborhood of such that
[TABLE]
Proof.
The proof is identical to that of Lemma 3.1, except for the estimate of in the second case, . Let us show that we can use Proposition 3.4 to handle this case. By the construction of the measure ,
[TABLE]
for all , cf. [3, Proposition 3.6]. We choose where and with , and . Then and so
[TABLE]
since is bounded. It follows by monotone convergence as that
[TABLE]
Therefore, and so
[TABLE]
On the other hand, we can estimate for the derivative of the convolution using (2.3), the distribution function of [42, Theorem 8.16] and the estimate . For a unit vector , it follows that
[TABLE]
As in Lemma 3.1, we conclude now from in the second case that . Thus . By Proposition 3.4, the right-hand side is in provided , which holds since . Thus we can use this as the bound for dominated convergence. The rest of the proof is as before. ∎
Remark 3.6*.*
If we consider a double phase functional in “normal” form, then the condition from the previous results can be written . This condition has proved to be of central importance when considering bounded solutions, cf. [6, 16, 32]. In this sense, the assumption in Lemma 3.5 is probably essentially sharp.
However, more precise research has established that one may even take for bounded minimizers [6, 33] (see also [21, 34] for the borderline case with unbounded minimizers). The borderline is handled using additional Hölder continuity obtained via De Giorgi technique, which in this case implies that for some . Indeed, from the previous proof we can see that would suffice if we had for some positive (as one has when ) instead of . However, for problems, such higher regularity of the function cannot be expected. Therefore, the borderline remains a problem for future research.
Let us also note that Ok [40] has considered double phase functionals under additional a priori integrability assumptions other than . If one could prove decay estimates for when , we could cover also this case. We do not know about such of results, so this, likewise, remains for a topic for another study.
4. Upper estimates for the double phase functional
The concept of -convergence, introduced by De Giorgi and Franzoni [23], has been systematically presented in [8, 19]. A family of functionals is said to -converge (in topology ) to if the following hold for every positive sequence converging to zero:
- (a)
for every and every -converging to ; 2. (b)
for every and some -converging to .
Let us remark that the somewhat strange assumption in the next theorem is actually quite natural: since is the set where the image edges occur, we cannot identify the edge if it coincides with the image boundary . On the other hand, we also have no need for the jump in the function at this location, since the other part of the jump will be outside the image, and thus cannot be seen.
Theorem 4.1**.**
Suppose that is a rectangular cuboid, , and assume that -a.e. on the boundary . Then -converges to in topology with .
Proof.
Let us start with condition (a) in the definition of -convergence. Let be a positive sequence converging to zero. Let and let be a sequence converging to in . If , then there is nothing to prove, so we assume that . We restrict our attention to a subsequence with and . Then is a bounded sequence in . By precompactness of there exists a limit function for a subsequence such that ; by reflexivity of and , we obtain subsequences with , in and . By in and the uniqueness of the limit, we conclude that .
The weak lower semi-continuity of the Lebesgue integral yields that
[TABLE]
and, since ,
[TABLE]
Finally, for the part we use the estimate from the previous paragraph, Young’s inequality and :
[TABLE]
By combining the above inequalities we obtain condition (a). Note that for this part we do not need the assumptions on and .
Let us then move to condition (b). Since is a rectangular cuboid, we can extend both the function and the weight by reflections to the rectangular cuboid with the same center but times the side-lengths. Then we use Lemma 3.1 with to conclude that there exist such that
[TABLE]
We need this inequality with instead of . Since and is a Sobolev function, . On the right-hand side, the same reason implies that
[TABLE]
The singular set of is contained in because . Since has Hausdorff -measure zero by assumption, it follows by the decomposition (2.1) that and so . Thus we have established condition (b) of -convergence. ∎
In the previous theorem we could consider a Lipschitz domain instead of a rectangular cuboid. In this case, the extension of both and would be done by flattening the boundary with the Lipschitz map. If we use Lemma 3.5 instead of Lemma 3.1, we obtain the following variant.
Theorem 4.2**.**
Suppose that is a bounded Lipschitz domain, for some , and assume that -a.e. on the boundary . Then -converges to in topology with .
We use the following formulation for relaxation, which emphasizes the connection with -convergence. A functional is the relaxation of in topology if
- (a)
for every and every -converging to ; 2. (b)
for every and some -converging to .
The relaxation is the greatest lower-semicontinuous minorant of . See [8, Proposition 1.31, p. 33]. Let us write for that
[TABLE]
We show that the relaxation of this functional equals . The proof is identical to Theorem 4.1, we simply take for every and . Naturally, we could also prove an analogue to Theorem 4.2.
Corollary 4.3**.**
Suppose that is a rectangular cuboid, , and assume that -a.e. on the boundary . Then in topology.
Acknowledgement
We thank the referee for some comments regarding this manuscript.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] M.K. Alaouia, T. Nabilab and M. Altanjia: On some new non-linear diffusion models for the image filtering, Applicable Anal. 93 (2014), no. 2, 269–280.
- 2[2] L. Ambrosio: Metric space valued functions of bounded variation. Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4) 17 (1990), no. 3, 439–478.
- 3[3] L. Ambrosio, N. Fusco and D. Pallara: Functions of Bounded Variation and Free Discontinuity Problems , Oxford Mathematical Monographs, Clarendon Press, Oxford University Press, New York, 2000.
- 4[4] G. Aubert and P. Kornprobst: Mathematical problems in image processing, Partial differential equations and the calculus of variations , Second edition, Applied Mathematical Sciences, vol. 147, Springer, New York, 2006.
- 5[5] P. Baroni, M. Colombo and G. Mingione: Harnack inequalities for double phase functionals, Nonlinear Anal. 121 (2015), 206–222.
- 6[6] P. Baroni, M. Colombo and G. Mingione: Regularity for general functionals with double phase, Calc. Var. Partial Differential Equations 57 (2018), no. 2, paper no. 62, 48 pp.
- 7[7] A. Benyaiche and I. Khlifi: Harnack inequality for quasilinear elliptic equations in generalized Orlicz-Sobolev spaces, Potential Anal. , to appear, DOI:10.1007/s 11118-019-09781-z.
- 8[8] A. Braides: Γ Γ \Gamma -convergence for beginners , Oxford Lecture Series in Mathematics and its Applications, vol. 22, Oxford University Press, Oxford, 2002, xii+218 pp.
