A theoretical guarantee for data completion via geometric separation

TL;DR

This paper provides a theoretical guarantee for data completion methods that leverage geometric separation, especially when data can be modeled as a mixture of different structural components.

Contribution

It introduces a unified theoretical framework that guarantees successful data recovery using combined separation and completion techniques for structured, incomplete data.

Findings

Theoretical proof of success for combined separation and completion methods.

Generalization of previous proofs to more complex data structures.

Applicable to data modeled as superpositions of multiple structures.

Abstract

Scientific and commercial data is often incomplete. Recovery of the missing information is an important pre-processing step in data analysis. Real-world data can in many cases be represented as a superposition of two or more different types of structures. For example, images may often be decomposed into texture and cartoon-like components. When incomplete data comes from a distribution well-represented as a mixture of different structures, a sparsity-based method combining concepts from data completion and data separation can successfully recover the missing data. This short note presents a theoretical guarantee for success of the combined separation and completion approach which generalizes proofs from the distinct problems.

Equations

$$\Vert \text{mathbbm}{1}_{{\Lambda }_1^c}{\Phi }_1{x}_1^0{\Vert }_1+\Vert \text{mathbbm}{1}_{{\Lambda }_2^c}{\Phi }_2{x}_2^0{\Vert }_1\le \delta .$$

$$\kappa$$

$$(x_1^{\star },x_2^{\star })=\underset{(x_1,x_2)}{\mathrm{argmin}}\Vert {\Phi }_1{x}_1{\Vert }_1+\Vert {\Phi }_2{x}_2{\Vert }_1\text{s.t.}{P}_K(x_1+x_2)=P_K(x^0).$$

$$\Vert x_1^{\star }-x_1^0{\Vert }_2+\Vert x_2^{\star }-x_2^0{\Vert }_2\le \frac{2\delta }{1-2\kappa }.$$

$$\Vert x_1^{\star }-x_1^0{\Vert }_2+\Vert x_2^{\star }-x_2^0{\Vert }_2$$

$$=$$

$$\le$$

$$=$$

$$\Vert \text{mathbbm}{1}_{{\Lambda }_1}{\Phi }_1(x_1^{\star }-x_1^0){\Vert }_1+\Vert \text{mathbbm}{1}_{{\Lambda }_1}{\Phi }_2(x_2^{\star }-x_2^0){\Vert }_1+\Vert \text{mathbbm}{1}_{{\Lambda }_1^c}{\Phi }_1(x_1^{\star }-x_1^0){\Vert }_1+\Vert \text{mathbbm}{1}_{{\Lambda }_2^c}{\Phi }_2(x_2^{\star }-x_2^0){\Vert }_1$$

$$=$$

$$+\Vert \text{mathbbm}{1}_{{\Lambda }_1^c}{\Phi }_1(P_M(x_1^0-x_1^{\star })+P_K(x_2^{\star }-x_2^0)){\Vert }_1+\Vert \text{mathbbm}{1}_{{\Lambda }_2^c}{\Phi }_2(x_2^{\star }-x_2^0){\Vert }_1,$$

$$\le$$

$$+\Vert \text{mathbbm}{1}_{{\Lambda }_1^c}{\Phi }_1(P_M(x_1^0-x_1^{\star })+P_K(x_2^{\star }-x_2^0)){\Vert }_1+\Vert \text{mathbbm}{1}_{{\Lambda }_2^c}{\Phi }_2(x_2^{\star }-x_2^0){\Vert }_1$$

$$=$$

$$\Vert {\Phi }_1(x_1^{\star }-x_1^0){\Vert }_1+\Vert {\Phi }_2(x_2^{\star }-x_2^0){\Vert }_1$$

$$\le \frac{1}{1-\kappa }\left(\Vert \text{mathbbm}{1}_{{\Lambda }_1^c}{\Phi }_1{x}_1^{\star }{\Vert }_1+\Vert \text{mathbbm}{1}_{{\Lambda }_2^c}{\Phi }_2{x}_2^{\star }{\Vert }_1+\delta \right).$$

$$\Vert \text{mathbbm}{1}_{{\Lambda }_1^c}{\Phi }_1{x}_1^{\star }{\Vert }_1+\Vert \text{mathbbm}{1}_{{\Lambda }_2^c}{\Phi }_2{x}_2^{\star }{\Vert }_1$$

$$=$$

$$\le$$

$$=$$

$$\le$$

$$\Vert {\Phi }_1(x_1^{\star }-x_1^0){\Vert }_1+\Vert {\Phi }_2(x_2^{\star }-x_2^0){\Vert }_1$$

$$\le$$

$$\Vert {\Phi }_1(x_1^{\star }-x_1^0){\Vert }_1+\Vert {\Phi }_2(x_2^{\star }-x_2^0){\Vert }_1\le {\left(1-\frac{\kappa }{1-\kappa }\right)}^{-1}\frac{2\delta }{1-\kappa }=\frac{2\delta }{1-2\kappa }.$$

Full text

For a Hilbert space $\mathcal{H}$ and coefficient index set $I$, a Parseval frame $\Phi:\mathcal{H}\rightarrow\ell^{2}(I)$ (or more precisely, the analysis operator of a Parseval frame) is such that $\left\|\Phi x\right\|_{2}=\left\|x\right\|_{\mathcal{H}}$ for all $x\in\mathcal{H}$. Data in a variety of applications of interest admit sparse representations with respect to a Parseval frame. It is known, for example, that if $\Phi$ is a shearlet frame for $L^{2}([0,1]^{2})$ and $x$ is a cartoon-like image, then there is some relatively “small” subset $\Lambda\subset I$ such that $\mathbbm{1}_{\Lambda}\Phi x$ captures “most” of the important information of $x$, where $\mathbbm{1}_{\Lambda}$ is the function that is $1$ on $\Lambda$ and [math] off of $\Lambda$ [1]. This sparsity can be used to perform image and data processing tasks including geometric separation of data into components with different fundamental structures [2, 3] and image inpainting or, more broadly, data recovery [2, 4, 5]. These two tasks can be combined. For example, by noting that natural images are usually a superposition of a cartoon-like part which is sparsely represented by a shearlet or curvelet frame and a textured part which is sparsely represented by a discrete cosine frame, one can use both frames simultaneously to perform empirically successful data completion [6, 7]. What follows is a generalization to the combined problem of the theoretical guarantees for data recovery [2, 4] and for data separation [2, 3].

[TABLE]

Let $P_{K}=P_{\mathcal{H}_{K}},P_{M}=P_{\mathcal{H}_{M}}$ be orthogonal projections onto $\mathcal{H}_{K},\mathcal{H}_{M}$, respectively, and let the joint concentration (with respect to $\Lambda_{1}$ and $\Lambda_{2}$) $\kappa$ be defined as

[TABLE]

Suppose $(x_{1}^{\star},x_{2}^{\star})$ solves the constrained optimization problem

[TABLE]

Then

[TABLE]

In Theorem 1.1, $\mathcal{H}_{K}$ represents the part of the data which is known, while $\mathcal{H}_{M}$ represents the missing part which is to be reconstructed. In the context of image inpainting, $\mathcal{H}_{K}$ represents the known image pixels, while $\mathcal{H}_{M}$ represents the unknown pixels to be recovered. $\Phi_{1}$ and $\Phi_{2}$ yield sparse representations of different types of structural information; for example, $\Phi_{1}$ could be a shearlet frame which sparsely represents cartoon-like images and $\Phi_{2}$ could sparsely represent texture. If $x^{0}$ is a superposition of structures which are sparsely represented by the $\Phi_{i}$, then relatively small index sets $\Lambda_{i}$ can be chosen yielding both a small $\delta$ and a small $\kappa$.

[TABLE]

Noting that $P_{K}(x_{1}^{\star}-x_{1}^{0})=-P_{K}(x_{2}^{\star}-x_{2}^{0})$ by the constraint in (1), $P_{M}(x_{1}^{0}-x_{1}^{\star})+P_{K}(x_{2}^{\star}-x_{2}^{0})=-(x_{1}^{\star}-x_{1}^{0})$, and hence:

[TABLE]

After some algebraic manipulation and employing $\delta$-sparsity, we see:

[TABLE]

So, it suffices to bound these terms involving $\mathbbm{1}_{\Lambda_{i}^{c}},i=1,2$. We note that $(x_{1}^{\star},x_{2}^{\star})$ is a minimizer of (1). Thus,

[TABLE]

Substituting (3) into (2), we obtain

[TABLE]

Thus,

[TABLE]

∎

Further theoretical results concerning the joint concentration and applications are the subjects of a forthcoming paper [8].

Acknowledgement

The authors met and began work on this project during the 2016 Hausdorff Trimester Program “Mathematics of Signal Processing” and are thus grateful to the organizers and the Hausdorff Institute for Mathematics in Bonn, Germany.