Bilinear Compressed Sensing under known Signs via Convex Programming

TL;DR

This paper introduces a convex programming method called $ ext{l}_1$-BranchHull for recovering two sparse vectors from their entrywise product when signs are known, with guarantees under noisy conditions and applications to real images.

Contribution

The paper proposes a novel convex program for bilinear inverse problems with known signs, providing recovery guarantees and practical algorithms for noisy and outlier scenarios.

Findings

Robust recovery with high probability under noise when $L \\geq \\Omega((S_1+S_2)\\log^{2}(K+N))$

Effective handling of noise and outliers through variants of $ ext{l}_1$-BranchHull

Successful extraction of piecewise constant signals from real images

Abstract

We consider the bilinear inverse problem of recovering two vectors, $\boldsymbol{x} \in\mathbb{R}^L$ and $\boldsymbol{w} \in\mathbb{R}^L$, from their entrywise product. We consider the case where $\boldsymbol{x}$ and $\boldsymbol{w}$ have known signs and are sparse with respect to known dictionaries of size $K$ and $N$, respectively. Here, $K$ and $N$ may be larger than, smaller than, or equal to $L$. We introduce $\ell_1$-BranchHull, which is a convex program posed in the natural parameter space and does not require an approximate solution or initialization in order to be stated or solved. Under the assumptions that $\boldsymbol{x}$ and $\boldsymbol{w}$ satisfy a comparable-effective-sparsity condition and are $S_1$- and $S_2$-sparse with respect to a random dictionary, we present a recovery guarantee in a noisy case. We show that $\ell_1$-BranchHull is robust to small dense noise with high probability if the number of measurements satisfy $L\geq\Omega\left((S_1+S_2)\log^{2}(K+N)\right)$. Numerical experiments show that the scaling constant in the theorem is not too large. We also introduce variants of $\ell_1$-BranchHull for the purposes of tolerating noise and outliers, and for the purpose of recovering piecewise constant signals. We provide an ADMM implementation of these variants and show they can extract piecewise constant behavior from real images.

Figures

Equations

$$\mathbf{y}=\mathbf{w}\odot \mathbf{x}\odot (1+\mathbf{\xi }),$$

$$\frac{\Vert {\mathbf{h}}^{♮}{\Vert }_1}{\Vert {\mathbf{h}}^{♮}{\Vert }_2}=\alpha \frac{\Vert {\mathbf{m}}^{♮}{\Vert }_1}{\Vert {\mathbf{m}}^{♮}{\Vert }_2}.$$

$${\xi }_{\ell }\ge -1\text{ for all }\ell =1,\dots ,L.$$

$$\displaystyle\text{$\ell_{1}$-BH}:\$$

$$\text{subject to}{s}_{\ell }({\mathbf{b}}_{\ell }^{⊺}\mathbf{h}{\mathbf{c}}_{\ell }^{⊺}\mathbf{m})\ge |y_{\ell }|$$

$$t_{\ell }{\mathbf{b}}_{\ell }^{⊺}\mathbf{h}\ge 0,\ell =1,2,\dots ,L.$$

$$\underset{c>0}{\min }\frac{{\Vert (\tilde{\mathbf{h}},\tilde{\mathbf{m}})-(c\hat{\mathbf{h}},c^{-1}\hat{\mathbf{m}})\Vert }_2}{\Vert (c\hat{\mathbf{h}},c^{-1}\hat{\mathbf{m}}){\Vert }_2}\le 37\sqrt{\Vert \mathbf{\xi }{\Vert }_{\infty }}$$

$${\ell }_1$$

$$\text{subject to }{s}_{\ell }({\mathbf{c}}_{\ell }^{⊺}\mathbf{m}+{\xi }_{\ell }){\mathbf{b}}_{\ell }^{⊺}\mathbf{h}\ge |y_{\ell }|,$$

$$t_{\ell }{\mathbf{b}}_{\ell }^{⊺}\mathbf{h}\ge 0,\ell =1,\dots ,L.$$

$$\text{TV BH}:$$

$$\text{subject to}{s}_{\ell }({\xi }_{\ell }+{\mathbf{c}}_{\ell }^{\top }\mathbf{m}){\mathbf{b}}_{\ell }^{\top }\mathbf{h}\ge |y_{\ell }|$$

$$t_{\ell }{\mathbf{b}}_{\ell }^{\top }\mathbf{h}\ge 0,\ell =1,2,\dots ,L.$$

$$\underset{\mathbf{h}\in {\mathbb{R}}^K,\mathbf{m}\in {\mathbb{R}}^N,\mathbf{\xi }\in {\mathbb{R}}^L}{\mathrm{minimize}}\Vert \mathbf{P}\mathbf{h}{\Vert }_1+\Vert \mathbf{m}{\Vert }_1+\lambda \Vert \mathbf{\xi }{\Vert }_1$$

$$\text{subject to}{s}_{\ell }({\xi }_{\ell }+{\mathbf{c}}_{\ell }^{\top }\mathbf{m}){\mathbf{b}}_{\ell }^{\top }\mathbf{h}\ge |y_{\ell }|$$

$$t_{\ell }{\mathbf{b}}_{\ell }^{\top }\mathbf{h}\ge 0,\ell =1,2,\dots ,L,$$

$$\mathbf{u}=\left(\begin{array}{c}\mathbf{x}\\ \mathbf{w}\\ \mathbf{\xi }\end{array}\right),\mathbf{v}=\left(\begin{array}{c}\mathbf{m}\\ \mathbf{h}\\ \lambda \mathbf{\xi }\end{array}\right).$$

$$\underset{\mathbf{v}\in {\mathbb{R}}^{N+K+L},\mathbf{u}\in {\mathbb{R}}^{3L}}{\mathrm{minimize}}\Vert \mathbf{Q}\mathbf{v}{\Vert }_1\text{subject to}\mathbf{u}=\mathbf{E}\mathbf{v},\mathbf{u}\in \mathcal{C}.$$

$$\underset{\mathbf{v},\mathbf{u},\mathbf{z}}{\mathrm{minimize}}\Vert \mathbf{z}{\Vert }_1\text{subject to}\mathbf{u}=\mathbf{E}\mathbf{v},\mathbf{Q}\mathbf{v}=\mathbf{z},\mathbf{u}\in \mathcal{C}.$$

$${\mathbf{u}}_{k+1}$$

$${\mathbf{z}}_{k+1}$$

$${\mathbf{v}}_{k+1}$$

$${\mathbf{\alpha }}_{k+1}$$

$${\mathbf{\beta }}_{k+1}$$

$${\mathbf{u}}_{k+1}=\text{mbox}{proj}_{\mathcal{C}}\left(\mathbf{E}{\mathbf{v}}_k-{\mathbf{\alpha }}_k\right),$$

$${\mathbf{z}}_{k+1}=S_{1/\rho }\left(\mathbf{Q}{\mathbf{v}}_k-{\mathbf{\beta }}_k\right),$$

$$\displaystyle\left(S_{c}(\boldsymbol{z})\right)_{i}=\left\{\begin{array}[]{cl}z_{i}-c&z_{i}>c\\ 0&|z_{i}|\leq c\\ z_{i}+c&z_{i}<-c\end{array}\right.,$$

$${\mathbf{v}}_{k+1}=$$

$$C=\left\{(\mathbf{x},\mathbf{w},\mathbf{\xi })\in {\mathbb{R}}^{3L}|s_{\ell }({\xi }_{\ell }+x_{\ell })w_{\ell }\ge |y_{\ell }|,t_{\ell }{w}_{\ell }\ge 0,\ell =1,\dots ,L\right\}$$

$$\underset{x\in \mathbb{R},w\in \mathbb{R},\xi \in \mathbb{R}}{\mathrm{argmin}}\frac{1}{2}{\Vert \left(\begin{array}{c}x\\ w\\ \xi \end{array}\right)-\left(\begin{array}{c}{x}_{\ell }^{\prime }\\ w_{\ell }^{\prime }\\ {\xi }_{\ell }^{\prime }\end{array}\right)\Vert }_2^{2}s.t.|y_{\ell }|-s_{\ell }xw-s_{\ell }\xi w\le 0,-t_{\ell }w\le 0.$$

$$\mathcal{L}(x,w,\xi ,{\mu }_1,{\mu }_2)=\frac{1}{2}{\Vert \left(\begin{array}{c}x\\ w\\ \xi \end{array}\right)-\left(\begin{array}{c}{x}_{\ell }^{\prime }\\ w_{\ell }^{\prime }\\ {\xi }_{\ell }^{\prime }\end{array}\right)\Vert }_2^2+{\mu }_1\left(|y_{\ell }|-s_{\ell }xw-s_{\ell }\xi w\right)-{\mu }_2\left(t_{\ell }w\right),$$

$$\frac{\partial \mathcal{L}}{\partial x}=x-x_{\ell }^{\prime }-{\mu }_1{s}_{\ell }w$$

$$\frac{\partial \mathcal{L}}{\partial w}=w-w_{\ell }^{\prime }-{\mu }_1{s}_{\ell }x-{\mu }_1{s}_{\ell }\xi -{\mu }_2{t}_{\ell }$$

$$\frac{\partial \mathcal{L}}{\partial \xi }=\xi -{\xi }_{\ell }^{\prime }-{\mu }_1{s}_{\ell }w$$

$${\mu }_1\ge 0,{\mu }_1\left(|y_{\ell }|-s_{\ell }xw-s_{\ell }\xi w\right)$$

$${\mu }_2\ge 0,{\mu }_2\left(t_{\ell }w\right)$$

$$|y_{\ell }|=s_{\ell }xw+s_{\ell }\xi w.$$

$$|y_{\ell }|=s_{\ell }\left(x_{\ell }^{\prime }+{\mu }_1{s}_{\ell }w\right)w+s_{\ell }\left({\xi }_{\ell }^{\prime }+{\mu }_1{s}_{\ell }w\right)w,$$

Full text

1 Introduction

We study the problem of recovering two unknown vectorss $\boldsymbol{x}$ and $\boldsymbol{w}$ in $\mathbb{R}^{L}$ from observations

[TABLE]

where $\odot$ denotes entrywise product and $\boldsymbol{\xi}\in\mathbb{R}^{L}$ is noise. Let $\boldsymbol{B}\in\mathbb{R}^{L\times K}$ and $\boldsymbol{C}\in\mathbb{R}^{L\times N}$ such that $\boldsymbol{w}=\boldsymbol{B}\boldsymbol{h}$ and $\boldsymbol{x}=\boldsymbol{C}\boldsymbol{m}$ with $\|\boldsymbol{h}\|_{0}\leq S_{1}$ and $\|\boldsymbol{m}\|_{0}\leq S_{2}$. The bilinear inverse problem (BIP) we consider is to find $\boldsymbol{w}$ and $\boldsymbol{x}$, up to the inherent scaling ambiguity, from $\boldsymbol{y}$, $\boldsymbol{B}$, $\boldsymbol{C}$ and $\operatorname{sign}{(\boldsymbol{w})}$.

BIPs, in general, have many applications in signal processing and machine learning and include fundamental practical problems like phase retrieval [Fienup, 1982, Candès and Li, 2012, Candès et al., 2013], blind deconvolution [Ahmed et al., 2014, Stockham et al., 1975, Kundur and Hatzinakos, 1996, Aghasi et al., 2016a], non-negative matrix factorization [Hoyer, 2004, Lee and Seung, 2001], self-calibration [Ling and Strohmer, 2015], blind source separation [O’Grady et al., 2005], dictionary learning [Tosic and Frossard, 2011], etc. These problems are in general challenging and suffer from identifiability issues that make the solution set non-unique and non-convex. A common identifiability issue, also shared by the BIP in (1), is the scaling ambiguity. In particular, if $(\boldsymbol{w}^{\natural},\boldsymbol{x}^{\natural})$ solves a BIP, then so does $(c\boldsymbol{w}^{\natural},c^{-1}\boldsymbol{x}^{\natural})$ for any nonzero $c\in\mathbb{R}$. In this paper, we resolve this scaling ambiguity by finding the point in the solution set closest to the origin with respect to the $\ell_{1}$ norm.

Another identifiability issue of the BIP in (1) is if $(\boldsymbol{w}^{\natural},\boldsymbol{x}^{\natural})$ solves (1), then so does $(\bf{1},\boldsymbol{w}^{\natural}\odot\boldsymbol{x}^{\natural})$, where $\bf{1}$ is the vector of ones. In prior works like Ahmed et al. [2014], this identifiability issue is resolved by assuming the unknown signals live in known subspaces. In contrast we resolve the identifiability issue by assuming the signals are sparse with respect to known bases or dictionaries. Natural choices for such bases include the standard basis, the Discrete Cosine Transform (DCT) basis, and a wavelet basis. In addition to sparsity of the unknown vectors, $\boldsymbol{w}^{\natural}$ and $\boldsymbol{x}^{\natural}$, with respect to known dictionaries, we also require sign information of $\boldsymbol{w}^{\natural}$ and $\boldsymbol{x}^{\natural}$. Knowing sign information is justified in, for example, imaging applications where $\boldsymbol{w}^{\natural}$ and $\boldsymbol{x}^{\natural}$ correspond to images, which contain non-negative pixel values.

In this paper, we study a sparse bilinear inverse problem, where the unknown vectors are sparse with respect to known dictionaries. Similar sparse BIPs have been extensively studied in the literature and are known to be challenging. In particular, the best known algorithm for sparse BIPs that can provably recovery $\boldsymbol{h}^{\natural}$ and $\boldsymbol{m}^{\natural}$ require measurements that scale quadratically, up to log factors, with respect to the sparsity levels, $S_{1}$ and $S_{2}$. Recent work on sparse rank-1 matrix recovery problem in Lee et al. [2017], which is motivated by considering the lifted version of the sparse blind deconvolution problem, provides an exact recovery guarantee of the sparse vectors $\boldsymbol{h}^{\natural}$ and $\boldsymbol{m}^{\natural}$ that satisfy a ‘peakiness’ condition, i.e. $\min\{\|\boldsymbol{h}^{\natural}\|_{\infty},\|\boldsymbol{m}^{\natural}\|_{\infty}\}\geq c$ for some absolute constant $c\in\mathbb{R}$, using a non-convex approach. This result holds with high probability for random measurements if the number of measurements satisfy $L\geq\Omega(S_{1}+S_{2})$, up to a log factor. For general vectors that are not constrained to a class of sparse vectors like those satisfying the peakiness condition, the same work shows exact recovery is possible if the number of measurements satisfy $L\geq\Omega(S_{1}S_{2})$, up to a log factor.

The main contribution of the present paper is to introduce a convex program in the natural parameter space for the sparse BIP described in (1) and show that it can stably recover sparse vectors, up to the global scaling ambiguity, provided they satisfy a comparable-effective-sparsity condition. Precisely, we say the sparse vectors $\boldsymbol{h}^{\natural}$ and $\boldsymbol{m}^{\natural}$ are $\rho$-comparable-effective-sparse, for some $\rho\geq 1$, if there exists an $\alpha\in\mathbb{R}$ that satisfies $\max\left(\alpha,1/\alpha\right)\leq\rho$ and

[TABLE]

Note that the ratio of the $\ell_{1}$ to $\ell_{2}$ norm of a vector is an approximate notion of the square root of the sparsity of a vector. Additionally, we assume the noise in (1) does not change the sign of the measurements. Specifically, we consider noise $\boldsymbol{\xi}\in\mathbb{R}^{L}$ such that

[TABLE]

Under the assumptions that the sparse vectorss satisfy (2) and the noise is small as in (3), we show that the convex program stably recovers the unknown vectors, up to the global scaling ambiguity, with high probability provided $\boldsymbol{B}$ and $\boldsymbol{C}$ are random and the number measurements satisfy $L\geq\Omega\left(S_{1}+S_{2}\right)$, up to log factors. Similar to the result in Lee et al. [2017], this results has optimal sample complexity, up to log factors, for a class of sparse signals, namely those with comparable sparsity levels.

1.1 Convex program and main results

We introduce a convex program written in the natural parameter space for the bilinear inverse problem described in (1). Let $(\boldsymbol{h}^{\natural},\boldsymbol{m}^{\natural})\in\mathbb{R}^{K}\times\mathbb{R}^{N}$ with $\|\boldsymbol{h}^{\natural}\|_{0}\leq S_{1}$ and $\|\boldsymbol{m}^{\natural}\|_{0}\leq S_{2}$. Let $w_{\ell}=\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}^{\natural}$, $x_{\ell}=\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m}^{\natural}$ and $y_{\ell}=\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}^{\natural}\cdot\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m}^{\natural}\cdot(1+\xi_{\ell})$, where $\boldsymbol{b}_{\ell}^{\intercal}$ and $\boldsymbol{c}_{\ell}^{\intercal}$ are the $\ell$th row of $\boldsymbol{B}$ and $\boldsymbol{C}$, respectively, and $\xi_{\ell}$ is the $\ell$th entry of $\boldsymbol{\xi}$. Also, let $\boldsymbol{s}=\operatorname{sign}(\boldsymbol{y})$ and $\boldsymbol{t}=\operatorname{sign}(\boldsymbol{w})=\operatorname{sign}(\boldsymbol{B}\boldsymbol{h}^{\natural})$. The convex program we consider to recover $(\boldsymbol{h}^{\natural},\boldsymbol{m}^{\natural})$ is the $\ell_{1}$-BranchHull program

[TABLE]

The motivation for the feasible set in program (4) follows from the observation that each measurement $y_{\ell}=w_{\ell}\cdot x_{\ell}$ defines a hyperbola in $\mathbb{R}^{2}$. As shown in Figure 3, the sign information $t_{\ell}=\operatorname{sign}(w_{\ell})$ restricts $(w_{\ell},x_{\ell})$ to one of the branches of the hyperbola. The feasible set in (4) corresponds to the convex hull of particular branches of the hyperbola for each $y_{\ell}$. This also implies that the feasible set is convex as it is the intersection of $L$ convex sets.

The objective function in (4) is an $\ell_{1}$ minimization over $(\boldsymbol{h},\boldsymbol{m})$ that finds a point $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ with $\|\hat{\boldsymbol{h}}\|_{1}=\|\hat{\boldsymbol{m}}\|_{1}$. Geometrically, this happens as the solution lies at the intersection of the $\ell_{1}$-ball and the hyperbolic curve (constraint) as shown in Figure 3. So, the minimizer of (4) in the noiseless case, under successful recovery, is $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}}):=\left(\boldsymbol{h}^{\natural}\sqrt{\frac{\|\boldsymbol{m}^{\natural}\|_{1}}{\|\boldsymbol{h}^{\natural}\|_{1}}},\boldsymbol{m}^{\natural}\sqrt{\frac{\|\boldsymbol{h}^{\natural}\|_{1}}{\|\boldsymbol{m}^{\natural}\|_{1}}}\right)$.

We now present our main result which states that the $\ell_{1}$-BranchHull program (4) stably recovers $\boldsymbol{w}$ and $\boldsymbol{x}$, up to the global scaling ambiguity, in the presence of small dense noise. We show that if $\boldsymbol{w}$ and $\boldsymbol{x}$ live in random subspaces with $\boldsymbol{h}^{\natural}$ and $\boldsymbol{m}^{\natural}$ containing at most $S_{1}$ and $S_{2}$ non zero entries, $\boldsymbol{h}^{\natural}$ and $\boldsymbol{m}^{\natural}$ satisfy comparable-effective-sparsity condition (2), the noise $\boldsymbol{\xi}$ satisfy (3), and there are at least $\Omega((S_{1}+S_{2})\log^{2}(K+N))$ number of measurements, then the minimizer of the $\ell_{1}$-BranchHull program is close to the bilinear ambiguity set $\{(c\hat{\boldsymbol{h}},c^{-1}\hat{\boldsymbol{m}})|c>0\}$ with high probability. Moreover, in the case noiseless case, the minimizer of the $\ell_{1}$-BranchHull is the point on this bilinear ambiguity set with equal $\ell_{1}$ norm with high probability, i.e. the minimizer is $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ with high probability.

Theorem 1** (Noisy recovery).**

Fix $\rho\geq 1$. Fix $(\boldsymbol{h}^{\natural},\boldsymbol{m}^{\natural})\in\mathbb{R}^{K+N}$ that are $\rho$-comparable-effective-sparse as defined in (2) with $\boldsymbol{h}^{\natural}\neq\bf{0}$, $\|\boldsymbol{h}^{\natural}\|_{0}\leq S_{1}$ and $\boldsymbol{m}^{\natural}\neq\bf{0}$, $\|\boldsymbol{m}^{\natural}\|_{0}\leq S_{2}$. Let $\boldsymbol{B}\in\mathbb{R}^{L\times K}$ and $\boldsymbol{C}\in\mathbb{R}^{L\times N}$ have i.i.d. $\mathcal{N}(0,1)$ entries. Let $\boldsymbol{y}\in\mathbb{R}^{L}$ contain measurements that satisfy (1) with noise $\boldsymbol{\xi}\in\mathbb{R}^{L}$ satisfying (3). If $L\geq C_{\rho}\left(\sqrt{S_{1}+S_{2}}\log(K+N)+t\right)^{2}$ for any $t>0$ then the $\ell_{1}$-BranchHull program (4) recovers $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})$ that satisfies

[TABLE]

with probability at least $1-\mathrm{e}^{-c_{t}L}$. Here, $C_{\rho}$ and $c_{t}$ are constants that depend quadratically on $\rho$ and $t$, respectively. Furthermore, $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})=(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ if $\boldsymbol{\xi}=\boldsymbol{0}$.

Theorem 1 shows that exact recovery, up to the global scaling ambiguity, of sparse vectors that satisfy the comparable-effective-sparsity condition is possible if the number of measurements satisfy $L\geq\Omega\left((S_{1}+S_{2})\log^{2}(K+N)\right)$. This result is optimal, up to the log factors. Numerical simulation on synthetic data verify Theorem 1 in the noiseless case and show that the constant in the sample complexity is not too large. We also present the results of numerical simulation on two real images which shows that a total variation reformulation of the convex program (4) can successfully recover the piecewise constant part of an otherwise distorted image.

1.2 Prior art for bilinear inverse problems

Recent approaches to solving bilinear inverse problems like blind deconvolution and phase retrieval include lifting the problems into a low rank matrix recovery task or to formulate a convex or non-convex optimization programs in the natural parameter space. Lifting transforms the problem of recovering $\boldsymbol{h}\in\mathbb{R}^{K}$ and $\boldsymbol{m}\in\mathbb{R}^{N}$ from bilinear measurements to the problem of recovering a low rank matrix $\boldsymbol{h}\boldsymbol{m}^{\intercal}$ from linear measurements. The low rank matrix can then be recovered using a semidefinite program. The result in Ahmed et al. [2014] for blind deconvolution showed that if $\boldsymbol{h}$ and $\boldsymbol{m}$ are coefficients of the target signals with respect to Fourier and Gaussian bases, respectively, then the lifting method successfully recovers the low rank matrix. The recovery occurs with high probability under near optimal sample complexity. Unfortunately, solving the semidefinite program is prohibitively computationally expensive because they operate in high-dimension space. Also, it is not clear whether or not it is possible to enforce additional structure like sparsity of $\boldsymbol{h}$ and $\boldsymbol{m}$ in the lifted formulation in a way that allows optimal sample complexity [Li and Voroninski, 2013, Oymak et al., 2015].

In comparison to the lifting approach for blind deconvolution and phase retrieval, methods that formulate an algorithm in the natural parameter space, such as alternating minimization and gradient descent based method, are computationally efficient and also enjoy rigorous recovery guarantees under optimal or near optimal sample complexity [Li et al., 2016, Candès et al., 2015, Netrapalli et al., 2013, Sun et al., 2016, Lee et al., 2017]. The work in Lee et al. [2017] for sparse blind deconvolution is based on alternating minimization. In the paper, the authors use an alternating minimization that successively approximate the sparse vectors while enforcing the low rank property of the lifted matrix. However, because these methods are non-convex, convergence to the global optimal requires a good initialization [Tu et al., 2015, Chen and Candes, 2015, Li et al., 2016].

Other approaches that operate in the natural parameter space include PhaseMax [Bahmani and Romberg, 2016, Goldstein and Studer, 2016] and BranchHull [Aghasi et al., 2016b]. PhaseMax is a linear program which has been proven to find the target signal in phase retrieval under optimal sample complexity if a good anchor vector is available. As with alternating minimization and gradient descent based approach, PhaseMax requires a good initializer to even be stated. In PhaseMax the initialization is part of the objective function but in alternating minimization the initialization is part of the algorithmic implementation. BranchHull is a convex program which solves the BIP described in (1) in the dense signal case under optimal sample complexity. Like the $\ell_{1}$-BranchHull presented in this paper, BranchHull does not require an initialization but does require the sign information of the signals.

The $\ell_{1}$-BranchHull program (4) combines strengths of both the lifting method and the gradient descent based method. Specifically, the $\ell_{1}$-BranchHull program is a convex program that operates in the natural parameter space, without a need for an initialization. These strengths are achieved at the cost of the sign information of the target signals $\boldsymbol{w}$ and $\boldsymbol{x}$, which can be justified in imaging applications where the goal might be to recover pixel values of a target image, which are non-negative.

1.3 Discussion and extensions

The $\ell_{1}$-BranchHull formulation is inspired by the BrachHull formation introduced in Aghasi et al. [2016b], which is a novel convex relaxation for the bilinear recovery of the entrywise product of vectors with known signs, and share many of its advantages and drawbacks. Like in BranchHull, $\ell_{1}$-BranchHull finds a point in the convex feasibility set that is closest to the origin. The important difference between these formulations is that BranchHull finds the point with the least $\ell_{2}$ norm while $\ell_{1}$-BranchHull finds the point with the least $\ell_{1}$-norm. Another difference is that BranchHull enjoys recovery guarantee when those vectors belong to random real subspaces while $\ell_{1}$-BranchHull enjoys recovery guarantee under a much weaker condition of those vectors admitting a sparse representation with respect to known dictionaries.

Similar to BranchHull, $\ell_{1}$-BranchHull is a flexible and can altered to tolerate large outliers. In this paper, we show that the $\ell_{1}$-BranchHull formulation is stable to small dense noise. However, as stated formulation (4) is not robust to outliers. This is because the formulations is particularly susceptible to noise that changes the sign of even a single measurement. For the bilinear inverse problem as described in (1) with small dense noise and arbitrary outliers, we propose the following robust $\ell_{1}$-BranchHull program by adding a slack variable.

[TABLE]

For measurements $y_{\ell}$ with incorrect sign, the corresponding slack variables $\xi_{\ell}$ shifts the feasible set so that the target signal is feasible. In the outlier case, the $\ell_{1}$ penalty promotes sparsity of slack variable $\boldsymbol{\xi}$. We implement a slight variation of the above program to remove distortions from an otherwise piecewise constant signal. In the case where $\boldsymbol{w}=\boldsymbol{B}\boldsymbol{h}^{\natural}$ is a piecewise constant signal, $\boldsymbol{x}=\boldsymbol{C}\boldsymbol{m}^{\natural}$ is a distortion signal and $\boldsymbol{y}=\boldsymbol{w}\odot\boldsymbol{x}$ is the distorted signal, the total variation version (6) of the robust BranchHull program (5), under successful recovery, produces the piecewise constant signal $\boldsymbol{B}\boldsymbol{h}^{\natural}$, up to a scaling.

[TABLE]

In (6), TV$(\cdot)$ is a total variation operator and is the $\ell_{1}$ norm of the vector containing pairwise difference of neighboring elements of the target signal $\boldsymbol{B}\boldsymbol{h}$. We implement (6) to remove distortions from images in Section 3 and leave detailed theoretical analysis of robust $\ell_{1}$ BranchHull (5) and its variant (6) to future work. It would also be interesting to develop convex relaxations in the natural parameter space that do not require sign information and to extend the analysis to the case when the phases of complex vectors are known and to the case of deterministic dictionaries instead of random dictionaries. All of these directions are left for future research.

1.4 Organization of the paper

The remainder of the paper is organized as follows. In Section 1.5, we present notations used throughout the paper. In Section 2, we present an Alternating Direction of Multipliers implementation of robust $\ell_{1}$-BranchHull program (5). In Section 3, we observe the performance of $\ell_{1}$-BranchHull on synthetic random data and natural images. In Section 4, we present the proof of Theorem 1.

1.5 Notation

Vectors and matrices are written with boldface, while scalars and entries of vectors are written in plain font. For example, $c_{\ell}$ is the $\ell$the entry of the vector $\boldsymbol{c}$. We write $\boldsymbol{1}$ as the vector of all ones with dimensionality appropriate for the context. We write $\boldsymbol{I}_{N}$ as the $N\times N$ identity matrix. For any $x\in\mathbb{R}$, let $(x)_{-}\in\mathbb{Z}$ such that $x-1<(x)_{-}\leq x$. For any matrix $\boldsymbol{A}$, let $\|\boldsymbol{A}\|_{F}$ be the Frobenius norm of $\boldsymbol{A}$. For any vector $\boldsymbol{x}$, let $\|\boldsymbol{x}\|_{0}$ be the number of non-zero entries in $\boldsymbol{x}$. For $\boldsymbol{x}\in\mathbb{R}^{K}$ and $\boldsymbol{y}\in\mathbb{R}^{N}$, $(\boldsymbol{x},\boldsymbol{y})$ is the corresponding vector in $\mathbb{R}^{K}\times\mathbb{R}^{N}$, and $\langle(\boldsymbol{x}_{1},\boldsymbol{y}_{1}),(\boldsymbol{x}_{2},\boldsymbol{y}_{2})\rangle=\langle\boldsymbol{x}_{1},\boldsymbol{x}_{2}\rangle+\langle\boldsymbol{y}_{1},\boldsymbol{y}_{2}\rangle$.

2 Algorithm

In this section, we present an Alternating Direction Method of Multipliers (ADMM) implementation of an extension of the robust $\ell_{1}$-BranchHull program (5). The ADMM implementation of the $\ell_{1}$-BranchHull program (4) is similar to the ADMM implementation of (7) and we leave it to the readers. The extension of the robust $\ell_{1}$-BranchHull program we consider is

[TABLE]

where $\boldsymbol{P}\in\mathbb{R}^{J\times K}$. The above extension reduces to the robust $\ell_{1}$-BranchHull program if $\boldsymbol{P}=\boldsymbol{I}_{K}$. Recalling that $\boldsymbol{w}=\boldsymbol{B}\boldsymbol{h}$ and $\boldsymbol{x}=\boldsymbol{C}\boldsymbol{m}$, we form the block diagonal matrices $\boldsymbol{E}=\mbox{diag}(\boldsymbol{C},\boldsymbol{B},\lambda^{-1}\boldsymbol{I}_{L})$ and $\boldsymbol{Q}=\mbox{diag}(\boldsymbol{I}_{N},\boldsymbol{P},\boldsymbol{I}_{L})$ and define the vectors

[TABLE]

Using this notation, our convex program can be compactly written as

[TABLE]

Here $\mathcal{C}=\{(\boldsymbol{x},\boldsymbol{w},\boldsymbol{\xi})\in\mathbb{R}^{3L}|\ s_{\ell}(\xi_{\ell}+x_{\ell})w_{\ell}\geq|y_{\ell}|,\ t_{\ell}w_{\ell}\geq 0,\ \ell=1,\dots,L\}$ is the convex feasible set of $\eqref{eq:Rl1BH}$. Introducing a new variable $\boldsymbol{z}$ the resulting convex program can be written as

[TABLE]

We may now form the scaled ADMM through the following steps

[TABLE]

which are followed by the dual updates

[TABLE]

We would like to note that the first three steps of the proposed ADMM scheme can be presented in closed form. The update in (8) is the following projection

[TABLE]

where $\mbox{proj}_{\mathcal{C}}(\boldsymbol{v})$ is the projection of $\boldsymbol{v}$ onto $\mathcal{C}$. Details of computing the projection onto $\mathcal{C}$ are presented in Appendix 2.1. The update in (9) can be written in terms of the soft-thresholding operator

[TABLE]

where for $c>0$,

[TABLE]

and $\left(S_{c}(\boldsymbol{z})\right)_{i}$ is the $i$th entry of $S_{c}(\boldsymbol{z})$. Finally, the update in (10) takes the following form

[TABLE]

In our implementation of the ADMM scheme, we initialize the algorithm with the $\boldsymbol{v}_{0}=\bf{0}$, $\boldsymbol{\alpha}_{0}=\bf{0}$, $\boldsymbol{\beta}_{0}=\bf{0}$.

2.1 Evaluation of the Projection Operator

Given a point $(\boldsymbol{x}^{\prime},\boldsymbol{w}^{\prime},{\boldsymbol{\xi}}^{\prime})\in\mathbb{R}^{3L}$, in this section we focus on deriving a closed-form expression for $\mbox{proj}_{\mathcal{C}}\left((\boldsymbol{x}^{\prime},\boldsymbol{w}^{\prime},{\boldsymbol{\xi}}^{\prime})\right)$, where

[TABLE]

is the convex feasible set of $\eqref{eq:Rl1BH}$. It is straightforward to see that the resulting projection program decouples into $L$ convex programs in $\mathbb{R}^{3}$ as

[TABLE]

Throughout this derivation we assume that $|y_{\ell}|>0$ (derivation of the projection for the case $y_{\ell}$ is easy) and as a result of which the second constraint $-t_{\ell}w\leq 0$ is never active (because then $w=0$ and the first constraint requires that $|y_{\ell}|\leq 0$). We also consistently use the fact that $t_{\ell}$ and $s_{\ell}$ are signs and nonzero.

Forming the Lagrangian as

[TABLE]

along with the primal constraints, the KKT optimality conditions are

[TABLE]

We now proceed with the possible cases.

Case 1. $\mu_{1}=\mu_{2}=0$:

In this case we have $(x,w,\xi)=(x_{\ell}^{\prime},w_{\ell}^{\prime},\xi_{\ell}^{\prime})$ and this result would only be acceptable when $|y_{\ell}|-s_{\ell}x_{\ell}^{\prime}w_{\ell}^{\prime}-s_{\ell}\xi_{\ell}^{\prime}w_{\ell}^{\prime}\leq 0$ and $t_{\ell}w_{\ell}^{\prime}\geq 0$.

Case 2. $\mu_{1}=0$, $t_{\ell}w=0$:

In this case the first feasibility constraint of (11) requires that $|y_{\ell}|\leq 0$, which is not possible when $|y_{\ell}|>0$.

Case 3. $|y_{\ell}|-s_{\ell}xw-s_{\ell}\xi w=0$, $t_{\ell}w=0$:

Similar to the previous case, this cannot happen when $|y_{\ell}|>0$.

Case 4. $\mu_{2}=0$, $|y_{\ell}|-s_{\ell}xw-s_{\ell}\xi w=0$:

In this case we have

[TABLE]

Now combining this observation with (12) and (14) yields

[TABLE]

and therefore

[TABLE]

Similarly, (13) yields

[TABLE]

Knowing that $w\neq 0$, $\mu_{1}$ can be eliminated between (17) and (19) to generate the following forth order polynomial equation in terms of $w$:

[TABLE]

After solving this 4-th order polynomial equation (e.g., the root command in MATLAB) we pick the real root $w$ which obeys

[TABLE]

Note that the second inequality in (20) warrants nonnegative values for $\mu_{1}$ thanks to (18). After picking the right root, we can explicitly obtain $\mu_{1}$ using (19) and calculate the solutions $x$ and $\xi$ using (12) and (14). Technically, in using the ADMM scheme for each $\ell$ we solve a forth-order polynomial equation and find the projection.

3 Numerical Experiments

In this section, we provide numerical experiments on synthetic random data and natural images where the signals follow the model in (1). We first show a phase portrait that verifies Theorem 1 in the noiseless case. Consider the following measurements: fix $N\in\{20,40,\dots,300\}$, $L\in\{4,8,\dots,140\}$ and let $K=N$. Let the target signal $(\boldsymbol{h}^{\natural},\boldsymbol{m}^{\natural})\in\mathbb{R}^{K}\times\mathbb{R}^{N}$ be such that both $\boldsymbol{h}^{\natural}$ and $\boldsymbol{m}^{\natural}$ have $0.05N$ non-zero entries with the nonzero indices randomly selected and set to $\pm 1$. Let $S_{1}$ and $S_{2}$ be the number of nonzero entries in $\boldsymbol{h}^{\natural}$ and $\boldsymbol{m}^{\natural}$, respectively. Let $\boldsymbol{B}\in\mathbb{R}^{L\times K}$ and $\boldsymbol{C}\in\mathbb{R}^{L\times N}$ such that $B_{ij}\sim\frac{1}{\sqrt{L}}\mathcal{N}(0,1)$ and $C_{ij}\sim\frac{1}{\sqrt{L}}\mathcal{N}(0,1)$. Lastly, let $\boldsymbol{y}=\boldsymbol{B}\boldsymbol{h}^{\natural}\odot\boldsymbol{C}\boldsymbol{m}^{\natural}$ and $\boldsymbol{t}=\text{sign}(\boldsymbol{B}\boldsymbol{h}^{\natural})$.

Figure 4 shows the fraction of successful recoveries from 10 independent trials using (4) for the bilinear inverse problem (1) from data as described above. Let $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ be the output of (4) and let $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})$ be the candidate minimizer. We solve (4) using an ADMM implementation similar to the ADMM implementation detailed in Section 2 with the step size parameter $\rho=1$. For each trial, we say (4) successfully recovers the target signal if $\|(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})-(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})\|_{2}<10^{-10}$. Black squares correspond to no successful recovery and white squares correspond to 100% successful recovery. The line corresponds to $L=C(S_{1}+S_{2})\log^{2}(K+N)$ with $C=0.25$ and indicates that the sample complexity constant in Theorem 1, in the noiseless case, is not very large.

We now show the result of using the total variation BranchHull program (6) to remove distortions from real images $\tilde{\boldsymbol{y}}\in\mathbb{R}^{p\times q}$. In the experiments, The observation $\boldsymbol{y}\in\mathbb{R}^{L}$ is the column-wise vectorization of the image $\tilde{\boldsymbol{y}}$, the target signal $\boldsymbol{w}=\boldsymbol{B}\boldsymbol{h}$ is the vectorization of the piecewise constant image and $\boldsymbol{x}=\boldsymbol{C}\boldsymbol{m}$ corresponds to the distortions in the image. We use (6) to recover piecewise constant target images like in the foreground of Figure 5(a) with TV$(\boldsymbol{B}\boldsymbol{h})=\|\boldsymbol{D}\boldsymbol{B}\boldsymbol{h}\|_{1}$, where \boldsymbol{D}=\left[\begin{array}[]{c}\boldsymbol{D}_{v}\\ \boldsymbol{D}_{h}\end{array}\right] in block form. Here, $\boldsymbol{D}_{v}\in\mathbb{R}^{(L-q)\times L}$ and $\boldsymbol{D}_{h}\in\mathbb{R}^{(L-p)\times L}$ with

[TABLE]

Lastly, we solve (6) using the ADMM algorithm detailed in Section 2 with $\boldsymbol{P}=\boldsymbol{D}\boldsymbol{B}$.

We use the total variation BranchHull program on two real images. The first image, shown in Figure 5(a), was captured using a camera and resized to a $115\times 115$ image. The measurement $\boldsymbol{y}\in\mathbb{R}^{L}$ is the vectorization of the image with $L=13225$. Let $\boldsymbol{B}$ be the $L\times L$ identity matrix. Let $\boldsymbol{F}$ be the $L\times L$ inverse DCT matrix. Let $\boldsymbol{C}\in\mathbb{R}^{L\times 300}$ with the first column set to $\bf{1}$ and remaining columns randomly selected from columns of $\boldsymbol{F}$ without replacement. The matrix $\boldsymbol{C}$ is scaled so that $\|\boldsymbol{C}\|_{F}=\|\boldsymbol{B}\|_{F}=\sqrt{L}$. The vector of known sign $\boldsymbol{t}$ is set to $\bf{1}$. Let $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}},\hat{\boldsymbol{\xi}})$ be the output of (6) with $\lambda=10^{3}$ and $\rho=10^{-4}$. Figure 5(b) corresponds to $\boldsymbol{B}\hat{\boldsymbol{h}}$ and shows that the object in the center was successfully recovered.

The second real image, shown in Figure 5(c), is an image of rice grains. The size of the image is $128\times 128$. The measurement $\boldsymbol{y}\in\mathbb{R}^{L}$ is the vectorization of the image with $L=16384$. Let $\boldsymbol{B}$ be the $L\times L$ identity matrix. Let $\boldsymbol{C}\in\mathbb{R}^{L\times 50}$ with the first column set to $\bf{1}$. The remaining columns of $\boldsymbol{C}$ are sampled from Bessel function of the first kind $J_{\nu}(\gamma)$ with each column corresponding to a fixed $\gamma\in\mathbb{R}$. Specifically, fix $\boldsymbol{g}\in\mathbb{R}^{L}$ with $g_{i}=-9+14\frac{i-1}{L-1}$. For each remaining column $\boldsymbol{c}$ of $\boldsymbol{C}$, fix $\boldsymbol{\zeta}\sim\mathcal{N}(\boldsymbol{0},\text{I}_{3})$ and let $c_{i}=J_{\frac{g_{i}}{6+0.1|\zeta_{1}|}+5|\zeta_{2}|}(0.1+10|\zeta_{3}|)$. The matrix $\boldsymbol{C}$ is scaled so that $\|\boldsymbol{C}\|_{F}=\|\boldsymbol{B}\|_{F}=\sqrt{L}$. The vector of known sign $\boldsymbol{t}$ is set to $\bf{1}$. Let $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}},\hat{\boldsymbol{\xi}})$ be the output of (6) with $\lambda=10^{3}$ and $\rho=10^{-7}$. Figure 5(d) corresponds to $\boldsymbol{B}\hat{\boldsymbol{h}}$.

4 Proof Outline

In this section, we provide a proof of Theorem 1 by considering a program similar to $\ell_{1}$-BrachHull program (4) with a different representation of the constraint set. Let $(\hat{w_{\ell}},\hat{x_{\ell}})=(\boldsymbol{b}_{\ell}^{\intercal}\hat{\boldsymbol{h}},\boldsymbol{c}_{\ell}^{\intercal}\hat{\boldsymbol{m}})$, $\alpha_{\ell}=\left(|\hat{w_{\ell}}|+|\hat{x_{\ell}}|\right)/2$ and define a convex function

[TABLE]

where $\gamma_{\ell}:\mathbb{R}^{2}\rightarrow\mathbb{R}_{>0}$ is a piecewise constant function such that

[TABLE]

Figures 8 and 8 show typical $f(w_{\ell},x_{\ell})$ for $\alpha_{\ell}\leq 1$ and $\alpha>1$, respectively.

Let $f_{\ell}(\boldsymbol{h},\boldsymbol{m})=f(\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h},\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m})$ with $\gamma_{\ell}(\boldsymbol{h},\boldsymbol{m}):=\gamma(\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h},\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m})$. We note that $f_{\ell}:\mathbb{R}^{K+N}\rightarrow\mathbb{R}$ is also a convex function because its epigraph is a convex set. The epigraph is convex because it is the inverse image of a convex set over a linear map. Define a one-sided loss function

[TABLE]

where $[\ \cdot\ ]_{+}$ denotes the positive side. We analyze the following generalized version of the $\ell_{1}$-BranchHull program:

[TABLE]

Program (24) is equivalent to the $\ell_{1}$-BranchHull in the sense that the objective and the constraint set of both the programs are the same. Lemma 1 shows that the set defined by constraints $s_{\ell}(\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m})\geq|y_{\ell}|$ with $t_{\ell}\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\geq 0$ and the set defined by constraints $f_{\ell}(\boldsymbol{h},\boldsymbol{m})\leq 0$ are the same set.

Lemma 1**.**

Fix $(\boldsymbol{h}^{\natural},\boldsymbol{m}^{\natural})\in\mathbb{R}^{K+N}$ such that $\boldsymbol{h}^{\natural}\neq 0$ and $\boldsymbol{m}^{\natural}\neq 0$. Let $\boldsymbol{B}\in\mathbb{R}^{L\times K}$, $\boldsymbol{C}\in\mathbb{R}^{L\times N}$ and $\boldsymbol{\xi}\in\mathbb{R}^{L}$. Let $\boldsymbol{y}\in\mathbb{R}^{L}$ contain measurements that satisfy (1). The set $\{(\boldsymbol{h},\boldsymbol{m})\in\mathbb{R}^{K+N}:s_{\ell}(\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m})\geq|y_{\ell}|,\ t_{\ell}\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\geq 0,\ \ell\in[L]\}$ is equal to the set $\{(\boldsymbol{h},\boldsymbol{m})\in\mathbb{R}^{K+N}:\mathcal{L}(\boldsymbol{h},\boldsymbol{m})\leq 0\}.$

Proof.

Fix an $\ell\in\{1,\dots,L\}$. It is sufficient to show that the set $\mathcal{S}_{\ell,1}:=\{(\boldsymbol{h},\boldsymbol{m})\in\mathbb{R}^{K+N}:s_{\ell}(\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m})\geq|y_{\ell}|,\ t_{\ell}\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\geq 0\}$ is equal to the set $\mathcal{S}_{\ell,2}:=\{(\boldsymbol{h},\boldsymbol{m})\in\mathbb{R}^{K+N}:f_{\ell}(\boldsymbol{h},\boldsymbol{m})\leq 0\}$. Consider a $(\boldsymbol{h},\boldsymbol{m})\in\mathcal{S}_{\ell,1}$. We have

[TABLE]

where (25) holds because $t_{\ell}\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\geq 0$ and $t_{\ell}s_{\ell}\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m}\geq 0$ and (26) holds because $\gamma_{\ell}(\boldsymbol{h},\boldsymbol{m})\geq 0$. Thus, $\mathcal{S}_{\ell,1}\subseteq\mathcal{S}_{\ell,2}$. Now, consider a $(\boldsymbol{h},\boldsymbol{m})\in\mathcal{S}_{\ell,2}$. W.L.O.G. assume $\gamma_{\ell}(\boldsymbol{h},\boldsymbol{m})\neq 0$. The reverse implications above implies $|y_{\ell}|\leq s_{\ell}\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m}$. Also, $t_{\ell}(\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}+s_{\ell}\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m})\geq 0$ because $4|y_{\ell}|+(\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}-s_{\ell}\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m})^{2}\geq 0$. If $\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}=0$, then $t_{\ell}\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\geq 0$. If $\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\neq 0$, then

[TABLE]

So, $t_{\ell}\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\geq 0$ because $s_{\ell}\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m}\geq|y_{\ell}|$ implies $\left(1+\frac{s_{\ell}\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m}}{\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}}\right)\geq 0$. Thus, $\mathcal{S}_{\ell,2}\subseteq\mathcal{S}_{\ell,1}$ as well, which proves that $\mathcal{S}_{\ell,1}=\mathcal{S}_{\ell,2}$ ∎

We will first show that if the noise $\boldsymbol{\xi}$ in the problem statement (1) satisfy $\xi_{\ell}\in[-1,0]$ for all $\ell\in\{1,\dots,L\}$, then the $\ell_{1}$-Generalized BranchHull program (24) recovers a point close to the set $\{(c\hat{\boldsymbol{h}},c^{-1}\hat{\boldsymbol{m}})|c>0\}$. We then extend the result to noise that satisfy condition (3). Since the $\ell_{1}$-Generalized BranchHull program (24) and the $\ell_{1}$-BranchHull program (4) are equivalent, the minimizer of $\ell_{1}$-BranchHull program is then also close to the set $\{(c\hat{\boldsymbol{h}},c^{-1}\hat{\boldsymbol{m}})|c>0\}$. Our strategy will be to show that for any feasible perturbation $(\delta\boldsymbol{h},\delta\boldsymbol{m})\in\mathbb{R}^{K+N}$, the objective of the $\ell_{1}$-Generalized BranchHull program (24) strictly increases outside a curved cylinder of radius $r$ centered at the bilinear ambiguity curve $\{(c\hat{\boldsymbol{h}},c^{-1}\hat{\boldsymbol{m}})|c>0\}$, where the radius depends on the level of noise.

The subgradient of the $\ell_{1}$-norm at $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ is

[TABLE]

where $\Gamma_{h},$ and $\Gamma_{m}$ denote the support of non-zeros in $\hat{\boldsymbol{h}}$, and $\hat{\boldsymbol{m}}$, respectively. We first consider the following descent direction that are orthogonal to the set $\mathcal{N}:=\text{span}(-\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$

[TABLE]

and show that descent direction from $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}$ of large $\ell_{2}$ norm is not feasible in (24). We do this by quantifying the “width" of the set $\mathcal{D}$ through a Rademacher complexity, and a probability that one of the subgradients of the constraint functions lie in a certain half space. In the noiseless case, we show that the solution of (24) is in the set $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}$. Since the only point in the set $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}$ consistent with the constraint of (24) is $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$, the minimizer of (24), in the noiseless case, is then $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$. In the noisy case, we use the boundedness of the feasible directions from the line $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}$ along with the observation that the feasible hyperbolic set diverges away from $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}$ to conclude the solution of minimizer of (24) is close to the bilinear ambiguity curve $\{(c\hat{\boldsymbol{h}},c^{-1}\hat{\boldsymbol{m}})|c>0\}$.

Recall that the constraint set of the $\ell_{1}$-BranchHull (4) has $L$ hyperbolic constraints and $L$ linear constraints. Let $\hat{\boldsymbol{y}}=\boldsymbol{B}\hat{\boldsymbol{h}}\odot\boldsymbol{C}\hat{\boldsymbol{m}}$ be the noiseless data. Using Lemma 1, each pair of hyperbolic and linear constraint $\{(\boldsymbol{h},\boldsymbol{m}):s_{\ell}\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m}\geq|\hat{y}_{\ell}|,\ t_{\ell}\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h}\geq 0\}$ can be expressed as $\hat{f}_{\ell}(\boldsymbol{h},\boldsymbol{m})\leq 0$ where

[TABLE]

First we note that a subgradient of $\hat{f}_{\ell}(\boldsymbol{h},\boldsymbol{m})$ at $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ is $\boldsymbol{z}_{\ell}:=-s_{\ell}(\boldsymbol{c}_{\ell}^{\intercal}\hat{\boldsymbol{m}}\boldsymbol{b}_{\ell}^{\intercal},\boldsymbol{b}_{\ell}^{\intercal}\hat{\boldsymbol{h}}\boldsymbol{c}_{\ell}^{\intercal})$. To see this, recall that $\alpha_{\ell}=\left(\boldsymbol{b}_{\ell}^{\intercal}\hat{\boldsymbol{h}}+\boldsymbol{c}_{\ell}^{\intercal}\hat{\boldsymbol{m}}\right)/2$ and let

[TABLE]

When $\alpha_{\ell}\leq 1$, we have $\hat{f}_{\ell}=\hat{g}_{\ell}$ because $\gamma_{\ell}(\boldsymbol{h},\boldsymbol{m})=\alpha_{\ell}$ by definition of $\gamma(\boldsymbol{b}_{\ell}^{\intercal}\boldsymbol{h},\boldsymbol{c}_{\ell}^{\intercal}\boldsymbol{m})=\gamma_{\ell}(\boldsymbol{h},\boldsymbol{m})$ in (23). When $\alpha_{\ell}>1$, $\hat{f}_{\ell}$ is non-differentiable at $(\boldsymbol{h},\boldsymbol{m})$ where $\hat{f}_{\ell}(\boldsymbol{h},\boldsymbol{m})=0$. Figure 8 shows the shape of $\hat{f}_{\ell}$ when $\alpha_{\ell}>1$. In this case, we have $\hat{f}_{\ell}(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})=0$ and $\hat{f}_{\ell}(\boldsymbol{h},\boldsymbol{m})=\hat{g}_{\ell}(\boldsymbol{h},\boldsymbol{m})$ for $(\boldsymbol{h},\boldsymbol{m})$ that satisfy $\hat{f}_{\ell}(\boldsymbol{h},\boldsymbol{m})\leq 0$. Thus, $\nabla\hat{g}_{\ell}(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\in\partial\hat{f}_{\ell}(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ and $-s_{\ell}(\boldsymbol{c}_{\ell}^{\intercal}\hat{\boldsymbol{m}}\boldsymbol{b}_{\ell}^{\intercal},\boldsymbol{b}_{\ell}^{\intercal}\hat{\boldsymbol{h}}\boldsymbol{c}_{\ell}^{\intercal})\in\partial\hat{f}_{\ell}(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ if $\nabla\hat{g}_{\ell}(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})=-s_{\ell}(\boldsymbol{c}_{\ell}^{\intercal}\hat{\boldsymbol{m}}\boldsymbol{b}_{\ell}^{\intercal},\boldsymbol{b}_{\ell}^{\intercal}\hat{\boldsymbol{h}}\boldsymbol{c}_{\ell}^{\intercal})$. So, consider

[TABLE]

where fourth equality holds because $t_{\ell}\boldsymbol{b}_{\ell}^{\intercal}\hat{\boldsymbol{h}}\geq 0$ and $t_{\ell}s_{\ell}\boldsymbol{c}_{\ell}^{\intercal}\hat{\boldsymbol{m}}\geq 0$ and the last equality holds because $\alpha_{\ell}=t_{\ell}(\boldsymbol{b}_{\ell}^{\intercal}\hat{\boldsymbol{h}}+s_{\ell}\boldsymbol{c}_{\ell}^{\intercal}\hat{\boldsymbol{m}})/2$. Similarly, we have $\frac{\partial\hat{g_{\ell}}}{\partial\boldsymbol{m}}(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})=-s_{\ell}\boldsymbol{b}_{\ell}^{\intercal}\hat{\boldsymbol{h}}\boldsymbol{c}_{\ell}$ and $\nabla\hat{g}_{\ell}(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})=-s_{\ell}(\boldsymbol{c}_{\ell}^{\intercal}\hat{\boldsymbol{m}}\boldsymbol{b}_{\ell}^{\intercal},\boldsymbol{b}_{\ell}^{\intercal}\hat{\boldsymbol{h}}\boldsymbol{c}_{\ell}^{\intercal})$. Define the Rademacher complexity of a set $\mathcal{D}\subset\mathbb{R}^{K+N}$ as

[TABLE]

where $\varepsilon_{1},\varepsilon_{2},\ldots,\varepsilon_{L}$ are iid Rademacher random variables independent of everything else. For a set $\mathcal{D}$, the quantity $\mathfrak{C}(\mathcal{D})$ is a measure of width of $\mathcal{D}$ around the origin in terms of the subgradients of the constraint functions. Our results also depend on a probability $p_{\tau}(\mathcal{D})$, and a positive parameter $\tau$ introduced below

[TABLE]

Intuitively, $p_{\tau}(\mathcal{D})$ quantifies the size of $\mathcal{D}$ through the subgradient vector. For a small enough fixed parameter, a small value of $p_{\tau}(\mathcal{D})$ means that the $\mathcal{D}$ is mainly invisible to the subgradient vector.

We now state a lemma which shows that if the noise $\boldsymbol{\xi}$ is such that

[TABLE]

and the number of measurements $L\geq\left(\frac{2\mathfrak{C}(\mathcal{D})+t\tau}{\tau\mathfrak{p}_{\tau}(\mathcal{D})}\right)^{2}$ for any $t>0$, then the solution of (24) is close to the bilinear ambiguity curve $\{(c\hat{\boldsymbol{h}},c^{-1}\hat{\boldsymbol{m}})|c>0\}$. The Proof of this lemma is based on small ball method developed in Koltchinskii and Mendelson [2015], Mendelson [2014] and further studied in Lecué et al. [2018], Lecué and Mendelson [2017], Bahmani and Romberg [2017].

Lemma 2**.**

Let $\mathcal{D}$ be the set of descent directions, already characterized in (28), for which $\mathfrak{C}(\mathcal{D})$, and $\mathfrak{p}_{\tau}(\mathcal{D})$ can be determined using (38), and (39). Let the noise $\boldsymbol{\xi}\in\mathbb{R}^{L}$ be such that (40) is satisfied. Choose $L\geq\left(\frac{2\mathfrak{C}(\mathcal{D})+t\tau}{\tau\mathfrak{p}_{\tau}(\mathcal{D})}\right)^{2}$ for any $t>0$. Then the solution $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})$ of the BH in (4) satisfies

[TABLE]

for some $\c{>}0$ with probability at least $1-\mathrm{e}^{-2Lt^{2}}$. Furthermore, $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})=(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ if $\boldsymbol{\xi}=\boldsymbol{0}$.

Proof.

Without loss of generality, we analyze the $\ell_{1}$-Generalized BranchHull program (24). We note that $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ is feasible in (24) because the noise satisfy $\boldsymbol{\xi}_{\ell}\in[-1,0]$ for all $\ell\in\{1,\dots,L\}$. We first control the set of feasible descent direction $(\delta\boldsymbol{h},\delta\boldsymbol{m})\in\mathcal{D}$ from the set $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}$. Since $(\delta\boldsymbol{h},\delta\boldsymbol{m})$ is a feasible perturbation from a point $(\boldsymbol{h}^{*},\boldsymbol{m}^{*}):=\left((1-\beta)\hat{\boldsymbol{h}},(1+\beta)\hat{\boldsymbol{m}}\right)$ for some $\beta\in\mathbb{R}$, we have from (24)

[TABLE]

Note that because of (41), $f_{\ell}(\boldsymbol{h}^{*}+\delta\boldsymbol{h},\boldsymbol{m}^{*}+\delta\boldsymbol{m})\leq 0$ for all $\ell\in\{1,\dots,L\}$ since, by definition, $\mathcal{L}(\boldsymbol{h},\boldsymbol{m})=\tfrac{1}{L}\sum_{\ell=1}^{L}\left[f_{\ell}(\boldsymbol{h},\boldsymbol{m})\right]_{+}$. Thus, $\gamma_{\ell}(\boldsymbol{h}^{*}+\delta\boldsymbol{h},\boldsymbol{m}^{*}+\delta\boldsymbol{m})$ in $f_{\ell}(\boldsymbol{h}^{*}+\delta\boldsymbol{h},\boldsymbol{m}^{*}+\delta\boldsymbol{m})$ satisfies $0\leq\gamma_{\ell}(\boldsymbol{h}^{*}+\delta\boldsymbol{h},\boldsymbol{m}^{*}+\delta\boldsymbol{m})\leq 1$. We now expand the loss function $\mathcal{L}(\boldsymbol{h},\boldsymbol{m})$ at $(\boldsymbol{h}^{*}+\delta\boldsymbol{h},\boldsymbol{m}^{*}+\delta\boldsymbol{m})$. Consider

[TABLE]

where in (42) we use $\operatorname{sign}(y_{\ell})=\operatorname{sign}(\hat{y}_{\ell})$ along with the fact that for $a\geq 0$ and $b<0$ with $a+b\geq 0$, we have $\sqrt{a+b}\geq\sqrt{a}-\sqrt{-b}$ and for $a\geq 0$ and $b\geq 0$, we have $\sqrt{a+b}\geq\sqrt{a}$. Also, (43) holds because $\hat{f}_{\ell}$ is convex with $\boldsymbol{z}_{\ell}\in\partial\hat{f}_{\ell}(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ and $0\leq\gamma_{\ell}\leq 1$. Lastly, (44) holds because $\langle\boldsymbol{z}_{\ell},(-\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\rangle=0$. Combining (41) and (44), we get

[TABLE]

where (45) follows from Cauchy-Schwartz inequality and (46) holds with probability $1-\mathrm{e}^{-cL}$ because, by Corollary 5.35 in Vershynin [2012], $\|\boldsymbol{B}\|\leq 3\sqrt{L}$ with probability $1-2\mathrm{e}^{-\tfrac{L}{2}}$. and $\|\boldsymbol{C}\|\leq 3\sqrt{L}$ with probability $1-2\mathrm{e}^{-\tfrac{L}{2}}$ as well. We now lower bound $I$ in (46). Let $\psi_{t}(s):=(s)_{+}-(s-t)_{+}$. Using the fact that $\psi_{t}(s)\leq(s)_{+}$, and that for every $\alpha,t\geq 0$, and $s\in\mathbb{R}$, $\psi_{\alpha t}(s)=t\psi_{\alpha}(\tfrac{s}{t})$, we have

[TABLE]

The proof mainly relies on lower bounding the right hand side above uniformly over all $(\delta\boldsymbol{h},\delta\boldsymbol{m})\in\mathcal{D}$. To this end, define a centered random process $\mathcal{R}(\boldsymbol{B},\boldsymbol{C})$ as follows

[TABLE]

and an application of bounded difference inequality McDiarmid [1989] yields that $\mathcal{R}(\boldsymbol{B},\boldsymbol{C})\leq\operatorname{E}\mathcal{R}(\boldsymbol{B},\boldsymbol{C})+t\tau/\sqrt{L}$ with probability at least $1-\mathrm{e}^{-2Lt^{2}}$. It remains to evaluate $\operatorname{E}\mathcal{R}(\boldsymbol{B},\boldsymbol{C})$, which after using a simple symmetrization inequality van der Vaart and Wellner [1997] yields

[TABLE]

where $\varepsilon_{1},\varepsilon_{2},\ldots,\varepsilon_{L}$ are independent Rademacher random variables. Using the fact that $\psi_{t}(s)$ is a contraction: $|\psi_{t}(\alpha_{1})-\psi_{t}(\alpha_{2})|\leq|\alpha_{1}-\alpha_{2}|$ for all $\alpha_{1},\alpha_{2}\in\mathbb{R}$, we have from the Rademacher contraction inequality Ledoux and Talagrand [2013] that

[TABLE]

In addition, using the facts that $t\mathbf{1}(s\geq t)\leq\psi_{t}(s)$ it follows

[TABLE]

Plugging (4), and (4) in (4), we have

[TABLE]

Combining this with (46), we obtain the final result

[TABLE]

Using the definitions in (38), and (39), we can write

[TABLE]

It is clear that choosing $L\geq\left(\tfrac{2\mathfrak{C}(\mathcal{D})+t\tau}{\tau\mathfrak{p}_{\tau}(\mathcal{D})}\right)^{2}$ implies the any feasible descent direction from $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}$ is bounded by

[TABLE]

with probability at least $1-\mathrm{e}^{-c_{t}L}$. Here $c_{t}$ is a constant that depends quadratically on $t$. Since $(\delta\boldsymbol{h},\delta\boldsymbol{m})\in\mathcal{N}_{\perp}$, the inequality above only gives us an element, $((1-\beta_{0})\hat{\boldsymbol{h}},(1+\beta_{0})\hat{\boldsymbol{m}})$ for some $\beta_{0}\in\mathbb{R}$, of the set $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}$ obeys

[TABLE]

That is, the solutions $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})$ cannot waver too far away from the line $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}$. We call this norm cylinder constraint as the solution must lie within a cylinder, centered at a line $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}$ and of radius given by the r.h.s. of the equation (53). Equivalently, a displacement $((1-\beta)\hat{\boldsymbol{h}},(1+\beta)\hat{\boldsymbol{m}})$ of the ground truth $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ is sufficiently close to $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})$. Using this fact together with the fact that the feasible hyperbolic set diverges away from the line $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}$ for large displacement $\beta$ and touches the line at $\beta=0$, we will conclude in the remaining proof that the Euclidean distance between $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})$ and the bilinear ambiguity curve corresponding to the ground truth $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ is bounded.

We first note that in the case when $\boldsymbol{\xi}=\boldsymbol{0}$, equation (53) implies that $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})$ must be on the line $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}_{\perp}$. Since the only element in the line $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}_{\perp}$ that is feasible is $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$, we conclude that in the noiseless case $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})=(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$.

Now, we use the fact that the noise $\boldsymbol{\xi}$ is such that $\xi_{\ell}\in[-1,0]$ for every $\ell\in[m]$, and there exists an $\ell^{\prime}\in[m]$ such that $\xi_{l^{\prime}}=0$. Trivially, the minimizer $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})$ must lie somewhere in the feasible set specified by the $\ell^{\prime}$ constraint: $\operatorname{sign}(y_{\ell})\boldsymbol{b}_{\ell^{\prime}}^{\intercal}\boldsymbol{h}\boldsymbol{c}_{\ell^{\prime}}^{\intercal}\boldsymbol{m}\geq|y_{\ell^{\prime}}|$ and $t_{\ell^{\prime}}\boldsymbol{b}_{\ell^{\prime}}^{\intercal}\boldsymbol{h}\geq 0$. Define the boundary $\mathcal{B}$ of the feasible set as follows

[TABLE]

The line $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})\oplus\mathcal{N}_{\perp}$ only touches the feasible set $\mathcal{B}$ at $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$. For a fixed displacement $\beta\in\mathbb{R}$ from $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$, define a segment of the norm cylinder in (53) as

[TABLE]

Clearly, there exists a $\beta_{0}$ such that $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})\in\mathcal{C}_{\beta_{0}}$. Moreover, there exist a $c>0$ such that $(c\hat{\boldsymbol{h}},\tfrac{1}{c}\hat{\boldsymbol{m}})\in\mathcal{C}_{\beta_{0}}$. This is because $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})$ must live in the convex hull of the set $\mathcal{B}$ and the bilinear ambiguity curve corresponding to $(\hat{\boldsymbol{h}},\hat{\boldsymbol{m}})$ is in the set $\mathcal{B}$. Since the distance between any two points in a cross-section of a cylinder is at most twice the radius of the cylinder, we have

[TABLE]

for some $c>0$.

∎

We now compute the Rademacher complexity $\mathfrak{C}(\mathcal{D})$ defined in (38) of the set of descent directions $\mathcal{D}$ defined in (28).

Lemma 3**.**

Fix $(\boldsymbol{h}^{\natural},\boldsymbol{m}^{\natural})\in\mathbb{R}^{K\times N}$. Let $\boldsymbol{B}\in\mathbb{R}^{L\times K}$ and $\boldsymbol{C}\in\mathbb{R}^{L\times N}$ have i.i.d. $\mathcal{N}(0,1)$ entries. Let $\mathcal{D}$ be as defined in (28). Then

[TABLE]

where $C>0$ is an absolute constant.

Proof.

We start by evaluating

[TABLE]

First note that on set $\mathcal{D}$, we have

[TABLE]

As for the remaining terms, we begin by writing

[TABLE]

and the second term in (4) is

[TABLE]

where the second inequality by the application of Lemma 5.2.2 in Akritas et al. [2016], and the final equality is due to the fact that $\|\boldsymbol{c}|_{\Gamma_{m}^{c}}\|_{\infty}^{2}$, and $\|\boldsymbol{b}|_{\Gamma_{h}^{c}}\|_{\infty}^{2}$ are subexponential and using Lemma 3 in van de Geer and Lederer [2013]. Plugging the bounds above back in (4), we obtain the upper bound on the Rademacher complexity given below

[TABLE]

∎

Next we compute the tail probability estimate $\mathfrak{p}_{\tau}(\mathcal{D})$ defined in (39).

Lemma 4**.**

Fix $(\boldsymbol{h}^{\natural},\boldsymbol{m}^{\natural})\in\mathbb{R}^{K\times N}$. Let $\boldsymbol{B}\in\mathbb{R}^{L\times K}$ and $\boldsymbol{C}\in\mathbb{R}^{L\times N}$ have i.i.d. $\mathcal{N}(0,1)$ entries. Let $\mathcal{D}$ be as defined in (28) and set $\tau=\sqrt{\tfrac{1}{2}\min\{\|\hat{\boldsymbol{h}}\|_{2}^{2},\|\hat{\boldsymbol{m}}\|_{2}^{2}\}}$ . Then $\mathfrak{p}_{\tau}(\mathcal{D})\geq\tfrac{1}{8c^{4}}$ for some absolute constant $c>0$.

Proof.

In order to evaulate

[TABLE]

it suffice to estimate the probability $\mathbb{P}(|\boldsymbol{b}_{\ell}^{\intercal}\hat{\boldsymbol{h}}\boldsymbol{c}_{\ell}^{\intercal}\delta\boldsymbol{m}+\boldsymbol{b}_{\ell}^{\intercal}\delta\boldsymbol{h}\boldsymbol{c}_{\ell}^{\intercal}\hat{\boldsymbol{m}}|\geq\tau)$. Using Paley-Zygmund inequality, we have

[TABLE]

Using norm equivalence of Gaussian random variables, we know that $\left(\operatorname{E}|\boldsymbol{b}_{\ell}^{\intercal}\hat{\boldsymbol{h}}\boldsymbol{c}_{\ell}^{\intercal}\delta\boldsymbol{m}+\boldsymbol{b}_{\ell}^{\intercal}\delta\boldsymbol{h}\boldsymbol{c}_{\ell}^{\intercal}\hat{\boldsymbol{m}}|^{4}\right)^{1/4}\leq c\left(\operatorname{E}|\boldsymbol{b}_{\ell}^{\intercal}\hat{\boldsymbol{h}}\boldsymbol{c}_{\ell}^{\intercal}\delta\boldsymbol{m}+\boldsymbol{b}_{\ell}^{\intercal}\delta\boldsymbol{h}\boldsymbol{c}_{\ell}^{\intercal}\hat{\boldsymbol{m}}|^{2}\right)^{1/2}$, this implies that

[TABLE]

Next, we show that $\operatorname{E}|\boldsymbol{b}_{\ell}^{\intercal}\hat{\boldsymbol{h}}\boldsymbol{c}_{\ell}^{\intercal}\delta\boldsymbol{m}+\boldsymbol{b}_{\ell}^{\intercal}\delta\boldsymbol{h}\boldsymbol{c}_{\ell}^{\intercal}\hat{\boldsymbol{m}}|^{2}\geq\ \min\{\|\hat{\boldsymbol{h}}\|_{2}^{2},\|\hat{\boldsymbol{m}}\|_{2}^{2}\}\left(\|\delta\boldsymbol{h}\|_{2}^{2}+\|\delta\boldsymbol{m}\|_{2}^{2}\right)$. Consider

[TABLE]

where the second and third equalities follow because $\boldsymbol{b}_{\ell}$ and $\boldsymbol{c}_{\ell}$ contain i.i.d. $\mathcal{N}(0,1)$ entries. The fourth equality follows from the fact $(\delta\boldsymbol{h},\delta\boldsymbol{m})\in\mathcal{D}\subset\mathcal{N}_{\perp}$, and hence $\mathcal{D}\perp\mathcal{N}$, which implies that $\delta\boldsymbol{h}^{\top}\hat{\boldsymbol{h}}=\delta\boldsymbol{m}^{\top}\hat{\boldsymbol{m}}.$ Normalizing by $\|(\delta\boldsymbol{h},\delta\boldsymbol{m})\|_{2}$, and comparing with (60) directly shows that $\tau^{2}=\tfrac{1}{2}\min\{\|\hat{\boldsymbol{h}}\|_{2}^{2},\|\hat{\boldsymbol{m}}\|_{2}^{2}\}$, and $\mathfrak{p}_{\tau}(\mathcal{D})\geq\tfrac{1}{8c^{4}}$. This completes the proof. ∎

We now present a proof of Theorem 1. In Theorem 1, the noise satisfy $\xi_{\ell}\geq-1$ which is in contrast to $\xi_{\ell}\in[-1,0]$ with $\xi_{\ell^{{}^{\prime}}}=0$ for some $\ell^{{}^{\prime}}\in\{1,\dots,L\}$ in Lemma 2. The key idea is measurements with noise that satisfy $\xi_{\ell}\geq-1$ can be converted to measurements with noise in the interval $[-1,0]$ with the noise for one of the measurement exactly equal to zero. In order to see this, let

[TABLE]

We then consider the measurements $y_{\ell}=s\hat{y_{\ell}}(1+\eta_{\ell})$ for $\ell\in[L]$. Because $s\hat{y_{\ell}}(1+\eta_{\ell})=\hat{y_{\ell}}(1+\xi_{\ell})$, the noisy measurements are the same, however the noise may be different.

Proof of Theorem 1.

As the noise of measurements $y_{\ell}=\hat{y_{\ell}}(1+\xi_{\ell})$ may not be one-sided as in (3), we consider equivalent measurements $y_{\ell}=s\hat{y_{\ell}}(1+\eta_{\ell})$, where $s$ and $\eta_{\ell}$ are as defined in (62) and (63), respectively. This turns the $\ell_{1}$-BranchHull program (4) into

[TABLE]

First, we note that for all $\ell\in\{1,\dots,L\}$,

[TABLE]

where the first inequality holds because $1+\xi_{\ell}\leq 1+\max_{\ell\in[L]}\xi_{\ell}\leq s$. Second, we have $\eta_{\ell}\geq-1$ for all $\ell\in\{1,\dots,L\}$, which follows directly from $\xi_{\ell}\geq-1$ for all $\ell$. Third, there exists a $\ell^{\prime}$ such that $\eta_{\ell^{\prime}}=0$. Thus, the noise $\boldsymbol{\eta}$ satisfies (3) and by Lemma 2, the minimizer $(\tilde{\boldsymbol{h}},\tilde{\boldsymbol{m}})$ of (64) is unique and if $L\geq\left(\tfrac{2\mathfrak{C}(\mathcal{D})+t\tau}{\tau\mathfrak{p}_{\tau}(\mathcal{D})}\right)^{2}$, the minimizer satisfies

[TABLE]

for some $c>0$ with probability at least $1-e^{-c_{t}L}$. Furthermore, $c\rightarrow 1$ as $\boldsymbol{\eta}\rightarrow\boldsymbol{0}$. In (68),

[TABLE]

where the first equality holds because $\eta_{\ell}\leq 0$ for all $\ell\in\{1,\dots,L\}$. We now compute

[TABLE]

where (70) holds because of triangle inequality, (71) holds because of (68), (72) holds because of (69), (73) holds because of (62) and (74) holds because for $\sqrt{1+\|\boldsymbol{\xi}\|_{\infty}}-1\leq\sqrt{\|\boldsymbol{\xi}\|_{\infty}}$. Lastly, we note that $\left(\tfrac{2\mathfrak{C}(\mathcal{D})+t\tau}{\tau\mathfrak{p}_{\tau}(\mathcal{D})}\right)^{2}\geq C_{\rho}\left(\sqrt{S_{1}+S_{2}}\log(K+N)+t\right)^{2}$ because of Lemmas 3, 4 and the assumption that $\boldsymbol{h}^{\natural}$ and $\boldsymbol{m}^{\natural}$ are $\rho$-comparable-effective-sparse as in (2), for some $\rho\geq 1$. Here, $C_{\rho}$ is constants that depends quadratically on $\rho$. This completes the proof. ∎