One-Bit Sensing of Low-Rank and Bisparse Matrices

TL;DR

This paper investigates the limits of recovering low-rank and bisparse matrices from one-bit measurements, providing theoretical error decay rates and practical algorithms for improved recovery accuracy.

Contribution

It introduces the concept of consistency width for error bounds and proposes a practical recovery algorithm with near-optimal decay rates for Gaussian sensing schemes.

Findings

Theoretical error decay rate is established using consistency width.

An idealized recovery method achieves fourth-root decay of optimal rate.

A practical algorithm attains sixth-root decay, close to optimal.

Abstract

This note studies the worst-case recovery error of low-rank and bisparse matrices as a function of the number of one-bit measurements used to acquire them. First, by way of the concept of consistency width, precise estimates are given on how fast the recovery error can in theory decay. Next, an idealized recovery method is proved to reach the fourth-root of the optimal decay rate for Gaussian sensing schemes. This idealized method being impractical, an implementable recovery algorithm is finally proposed in the context of factorized Gaussian sensing schemes. It is shown to provide a recovery error decaying as the sixth-root of the optimal rate.

Equations

$${\Sigma}_{(s)}^{[r]}:=\big{\{}{\bm{X}}\in\mathbb{R}^{n\times n}:\operatorname{rank}({\bm{X}})\leq r,\,\exists S,T\subseteq\llbracket n\rrbracket,\\ |S|=|T|=s,\,{\bm{X}}_{\overline{S}\times\llbracket n\rrbracket}={\bm{0}},\,{\bm{X}}_{\llbracket n\rrbracket\times\overline{T}}={\bm{0}}\big{\}}.$$

$$y_i=\mathrm{sgn}(⟨{\mathbf{A}}_i,\mathbf{X}{⟩}_F)=\mathrm{sgn}(\mathrm{tr}({\mathbf{A}}_i^{\top }\mathbf{X})),i\in [[m]].$$

$$\begin{gathered} {\mathrm{CW}}^m(\mathcal{K}):=\underset{\mathcal{A}:{\mathbb{R}}^{n\times n}\to {\mathbb{R}}^m}{\inf }\sup \{\Vert \mathbf{X}-{\mathbf{X}}^{\prime }{\Vert }_F:\mathbf{X},{\mathbf{X}}^{\prime }\in \mathcal{K}, \\ \mathrm{sgn}\mathcal{A}(\mathbf{X})=\mathrm{sgn}\mathcal{A}({\mathbf{X}}^{\prime })\}. \end{gathered}$$

$$c\,\frac{rs}{m}\ \leq\ \operatorname{CW}^{m}\big{(}\widetilde{\Sigma}_{(s)}^{[r]}\big{)}\ \leq\ C\,\frac{rs}{m}\ \ln\left(\frac{nm}{rs}\right).$$

$$\mathcal{E}^{m}(\mathcal{K}):=\inf_{(\bm{\mathcal{A}},\bm{\Delta})}\sup_{\bm{X}\in\mathcal{K}}\|\bm{X}-\bm{\Delta}\big{(}\operatorname{sgn}\bm{\mathcal{A}}(\bm{X})\big{)}\|_{F}.$$

$${\mathcal{E}}^m(\mathcal{K})\le {\mathrm{CW}}^m(\mathcal{K})\le 2{\mathcal{E}}^m(\mathcal{K}).$$

$$\mathcal{E}^{m}\big{(}\widetilde{\Sigma}_{(s)}^{[r]}\big{)}\leq\varepsilon\ \Longrightarrow m\geq C\varepsilon^{-1}rs.$$

$$\mathcal{H}(\mathcal{K},\eta ):=\ln \inf \{|\mathcal{G}|:\mathcal{G}\subseteq \mathcal{K}\subseteq \mathcal{G}+\eta {\mathbb{B}}_F^{n\times n}\},$$

$$\textstyle m\geq\frac{C}{\varepsilon}\mathcal{H}\big{(}\mathcal{K},c\,\varepsilon\big{)},$$

$$\mathrm{sgn}\mathcal{A}(\mathbf{X})=\mathrm{sgn}\mathcal{A}({\mathbf{X}}^{\prime })⟹\Vert \mathbf{X}-{\mathbf{X}}^{\prime }{\Vert }_F\le \epsilon .$$

$$|\frac{1}{m}\Vert \mathrm{sgn}\mathcal{A}\mathbf{X},\mathrm{sgn}\mathcal{A}{\mathbf{X}}^{\prime }{\Vert }_H-\frac{1}{\pi }\arccos ⟨\mathbf{X},{\mathbf{X}}^{\prime }⟩|\ge c\epsilon ,$$

$$\textstyle m\geq\frac{C}{\varepsilon}\mathcal{H}\big{(}\mathcal{K},c\,\frac{\varepsilon}{n}\big{)}.$$

$$\mathbb{P}_{0}:=\small\mathbb{P}\big{(}\exists{\bm{X}},{\bm{X}}^{\prime}\!\in\mathcal{K}:\|{\bm{X}}-{\bm{X}}^{\prime}\|_{F}>\varepsilon,\ \operatorname{sgn}\bm{\mathcal{A}}({\bm{X}})=\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}}^{\prime})\big{)}.$$

$$\displaystyle\mathbb{P}_{0}\leq\mathbb{P}\big{(}$$

$$\displaystyle\exists\bm{Z},\bm{Z}^{\prime}\in\rho\mathbb{B}^{n\times n}_{F}:\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}}_{k}+{\bm{Z}})=\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}}_{k^{\prime}}+{\bm{Z}}^{\prime})\big{)}.$$

$${\mathbb{P}}_0\le {\sum }_{(k,k^{\prime })\in \Omega }{\mathbb{P}}_{k,k^{\prime }},$$

$${\mathbb{P}}_{k,k^{\prime }}$$

$$\displaystyle\small=\mathbb{P}\big{(}\exists\bm{Z},\bm{Z}^{\prime}\in\rho\mathbb{B}^{n\times n}_{F}:\forall i\in\llbracket m\rrbracket,$$

$$\displaystyle\small\hskip 25.60747pt\operatorname{sgn}(\langle{\bm{A}}_{i},{\bm{X}}_{k}+{\bm{Z}}\rangle_{F})=\operatorname{sgn}(\langle{\bm{A}}_{i},{\bm{X}}_{k^{\prime}}+{\bm{Z}}^{\prime}\rangle_{F})\big{)}.$$

$${\mathbb{P}}_{k,k^{\prime }}$$

$$\displaystyle\hskip 25.60747pt\operatorname{sgn}(\langle{\bm{A}}_{i},{\bm{X}}_{k}+{\bm{Z}}\rangle_{F})=\operatorname{sgn}(\langle{\bm{A}}_{i},{\bm{X}}_{k^{\prime}}+{\bm{Z}}^{\prime}\rangle_{F})\big{)}$$

$$\displaystyle\textstyle=\prod_{i=1}^{m}\mathbb{P}\big{(}\exists\bm{Z},\bm{Z}^{\prime}\in\rho\mathbb{B}^{n\times n}_{F}:$$

$$\displaystyle\hskip 25.60747pt\operatorname{sgn}(\langle{\bm{A}}_{i},{\bm{X}}_{k}+{\bm{Z}}\rangle_{F})=\operatorname{sgn}(\langle{\bm{A}}_{i},{\bm{X}}_{k^{\prime}}+{\bm{Z}}^{\prime}\rangle_{F})\big{)}$$

$$\displaystyle\textstyle\leq\prod_{i=1}^{m}\Big{(}1-\frac{1}{\pi}\arccos(\langle{\bm{X}}_{k},{\bm{X}}_{k^{\prime}}\rangle_{F})+\sqrt{\frac{\pi}{2}}n\rho\Big{)},$$

$$\textstyle\mathbb{P}_{k,k^{\prime}}\leq\prod_{i=1}^{m}\big{(}1-\frac{\varepsilon}{2\pi}\big{)}\leq\exp(-\frac{\varepsilon}{2\pi}m).$$

$${\mathbb{P}}_0\le K^2\exp (-\frac{\epsilon }{2\pi }m)=\exp (2\mathcal{H}(\mathcal{K},c\frac{\epsilon }{n})-\frac{\epsilon }{2\pi }m),$$

$$\displaystyle\mathcal{H}\big{(}\widetilde{\Sigma}_{(s)}^{[r]},\eta)$$

$$\displaystyle\leq Crs\ln\big{(}n/(\eta s)\big{)}.$$

$$m\ge \frac{C}{\epsilon }rs\ln (\frac{n}{\epsilon s}),$$

$$\underset{\begin{array}{c}\mathbf{X},{\mathbf{X}}^{\prime }\in {\tilde{\Sigma }}_{(s)}^{[r]}\\ \mathrm{sgn}\mathcal{A}(\mathbf{X})=\mathrm{sgn}\mathcal{A}({\mathbf{X}}^{\prime })\end{array}}{\sup }\Vert \mathbf{X}-{\mathbf{X}}^{\prime }{\Vert }_F\le \epsilon .$$

$$\epsilon =\frac{C}{m}rs\ln (\frac{n}{\epsilon s})\ge C\frac{rs}{m},\text{mbox}andinturn\epsilon \le \frac{C}{m}rs\ln (\frac{cnm}{rs}),$$

$${\mathbf{X}}^{\prime }={\mathcal{P}}_{{\Sigma }_{(s)}^{[r]}}({\mathcal{A}}^{\ast }\mathbf{y}).$$

$$m\ge \frac{C}{{\delta }^2}rs\ln (\frac{n}{s}),$$

$$(1-\delta )\Vert \mathbf{Z}{\Vert }_F\le \frac{\sqrt{\pi /2}}{m}\Vert \mathcal{A}(\mathbf{Z}){\Vert }_1\le (1+\delta )\Vert \mathbf{Z}{\Vert }_F.$$

Full text

I Introduction

We are interested in matrices that are simultaneously low-rank and bisparse, i.e., anticipating over some notation to be introduced momentarily, matrices in the set

[TABLE]

Determining the minimal number of linear measurements allowing for uniform stable recovery of matrices in ${\Sigma}_{(s)}^{[r]}$ is a difficult problem, e.g. [1] contains negative results for optimization-based recovery, while [2] showed, with some caveats, that $\Theta(rs\ln(n/s))$ measurement suffice. Here, we consider a similar problem for quantized linear measurements, hoping for more favorable conclusions in this case. Precisely, suppose that matrices ${\bm{X}}\in{\Sigma}_{(s)}^{[r]}$ are acquired via the one-bit measurements

[TABLE]

We ask: how large should $m$ be to make it possible to derive, from ${\bm{y}}\in\{\pm 1\}^{m}$, an approximant ${\bm{X}}^{\prime}$ such that $\|{\bm{X}}-{\bm{X}}^{\prime}\|_{F}\leq\varepsilon$, or, turning things around, how fast can $\|{\bm{X}}-{\bm{X}}^{\prime}\|_{F}$ decay when $m$ increases? Phrased in terms of consistency width, Section II addresses this question with lower and upper estimates that almost match. A (weaker) upper estimate also appears in Section III, which introduces an idealized recovery scheme not enforcing consistency of the approximant. This idealized scheme being impractical (probably NP-hard), we exhibit in Section IV a multistep recovery scheme working with specific sensing schemes.

Notation: Given ${\bm{M}}\in\mathbb{R}^{n\times n}$, ${\bm{M}}_{\Omega_{1}\times\Omega_{2}}$ represents the submatrix of ${\bm{M}}$ indexed by $\Omega_{1}\times\Omega_{2}\subseteq\llbracket n\rrbracket\times\llbracket n\rrbracket$, with $\llbracket n\rrbracket:=\{1,\ldots,n\}$. The set $\overline{\Omega}\subseteq\llbracket n\rrbracket$ stands for $\llbracket n\rrbracket\setminus\Omega$. The Frobenius norm is denoted by $\|\cdot\|_{F}$ and the Frobenius inner product by $\langle\cdot,\cdot\rangle_{F}$. Given index sets $S,T\subseteq\llbracket n\rrbracket$, the set ${\Sigma}_{S\times T}^{[r]}$ contains all rank-$r$ matrices supported on $S\times T$. A tilde over a set $\mathcal{K}\subseteq\mathbb{R}^{n\times n}$, e.g. $\widetilde{\Sigma}_{(s)}^{[r]}$ or $\widetilde{\Sigma}_{S\times T}^{[r]}$, means that this set is intersected with the sphere $\mathbb{S}_{F}^{n\times n}$ of $\mathbb{R}^{n\times n}$ with respect to the Frobenius norm. The notation for the unit ball is $\mathbb{B}_{F}^{n\times n}$.

We often rewrite (1) in more compact form by defining the map \bm{\mathcal{A}}:{\bm{M}}\in\mathbb{R}^{n\times n}\mapsto\big{(}\langle{\bm{A}}_{i},{\bm{M}}\rangle_{F}\big{)}_{i=1}^{m}\in\mathbb{R}^{m}, so that (1) also reads ${\bm{y}}=\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}})$ with $\operatorname{sgn}\bm{\mathcal{A}}:=\operatorname{sgn}\circ\,\bm{\mathcal{A}}$. The sign operator $\operatorname{sgn}$ acts entrywise when applied to a vector. We say that $\bm{\mathcal{A}}$ is a Gaussian random map when the matrices $\bm{A}_{1},\ldots,\bm{A}_{m}$ are independent $n\times n$ Gaussian random matrices, i.e., $(\bm{A}_{i})_{k,l}\sim_{\mathrm{i.i.d.}}\mathcal{N}(0,1)$. The adjoint of $\bm{\mathcal{A}}$ is the map $\bm{\mathcal{A}}^{*}:\bm{v}\in\mathbb{R}^{m}\to\sum_{i}v_{i}\bm{A}_{i}\in\mathbb{R}^{n\times n}$. All absolute constants are denoted by $C,C^{\prime},c,c^{\prime},\ldots$ and their values can change from line to line.

II Consistency width

The $m^{\rm th}$ consistency width of a set $\mathcal{K}\subseteq\mathbb{S}_{F}^{n\times n}$ is defined as

[TABLE]

Essentially, $\operatorname{CW}^{m}(\mathcal{K})$ provides the best (with respect to $\bm{\mathcal{A}}$) worst-case error of any consistent recovery scheme, i.e., one that outputs $\bm{X}^{\prime}\in\mathcal{K}$ with the same one-bit measurements as the sensed matrix $\bm{X}\in\mathcal{K}$, see [3]. We shall show that

[TABLE]

II-A Lower estimate

Let us first establish the lower bound in (2). Given $\mathcal{K}\subseteq\mathbb{S}_{F}^{n\times n}$, we define the smallest achievable worst-case error of any combination of one-bit maps $\operatorname{sgn}\bm{\mathcal{A}}$ with recovery schemes $\bm{\Delta}:\{\pm 1\}^{m}\to\mathcal{K}$ by

[TABLE]

Note that $\bm{\Delta}$ is not required to output a consistent approximant. However, the smallest achievable worst-case errors using consistent or unrestricted schemes are in fact equivalent, as emphasized below.

Proposition 1**.**

For any $\mathcal{K}\subseteq\mathbb{S}_{F}^{n\times n}$, one has

[TABLE]

Proof.

Given $(\bm{\mathcal{A}},\bm{\Delta})$ and $\bm{X},\bm{X}^{\prime}\in\mathcal{K}$ sharing the same one-bit measurements, in view of $\bm{\Delta}(\operatorname{sgn}\bm{\mathcal{A}}(\bm{X}))=\bm{\Delta}(\operatorname{sgn}\bm{\mathcal{A}}(\bm{X}^{\prime}))$, there holds $\|\bm{X}-\bm{X}^{\prime}\|_{F}\leq\|\bm{X}-\bm{\Delta}(\operatorname{sgn}\bm{\mathcal{A}}(\bm{X}))\|_{F}+\|\bm{X}^{\prime}-\bm{\Delta}(\operatorname{sgn}\bm{\mathcal{A}}(\bm{X}^{\prime}))\|_{F}$, which yields the rightmost inequality in (3).

Now, given $\bm{\mathcal{A}}$ and $\bm{X}\in\mathcal{K}$, we choose $\bm{\Delta}(\operatorname{sgn}\bm{\mathcal{A}}(\bm{X}))$ as an arbitrary matrix $\bm{X}^{\prime}\in\mathcal{K}$ consistent with ${\bm{X}}$. The leftmost inequality in (3) follows easily. ∎

Proposition 1 implies in particular that the lower bound in (2) can be derived from the following lower estimation of $\mathcal{E}^{m}(\widetilde{\Sigma}_{(s)}^{[r]})$.

Theorem 1**.**

For an error level $\varepsilon>0$ , one has

[TABLE]

Equivalently, this means that \mathcal{E}^{m}\big{(}\widetilde{\Sigma}_{(s)}^{[r]}\big{)}\geq C\frac{rs}{m}.

Proof.

Suppose that \mathcal{E}^{m}\big{(}\widetilde{\Sigma}_{(s)}^{[r]}\big{)}\leq\varepsilon. We fix subsets $S,T$ of $\llbracket n\rrbracket$ with $|S|=|T|=s$, a matrix $\bm{M}\in\mathbb{R}^{n\times r}$ such that $\bm{M}_{\overline{S}\times\llbracket r\rrbracket}=\bm{0}$, and consider the $rs$-dimensional subspace $\mathcal{S}$ of $\mathbb{R}^{n\times n}$ defined by $\mathcal{S}:=\{\bm{M}\bm{N}:\bm{N}\in\mathbb{R}^{r\times n},\ \bm{N}_{\,\llbracket r\rrbracket\times\overline{T}}=\bm{0}\}$. Note that $\mathcal{S}\subseteq{\Sigma}_{(s)}^{[r]}$, hence $\mathcal{E}^{m}(\widetilde{\mathcal{S}})\leq\mathcal{E}(\widetilde{\Sigma}_{(s)}^{[r]})\leq\varepsilon$. Therefore, for any $\varepsilon^{\prime}>\varepsilon$, say $\varepsilon^{\prime}=2\varepsilon$, we can find a pair $(\bm{\mathcal{A}},\bm{\Delta})$ such that \|\bm{X}-\bm{\Delta}\big{(}\operatorname{sgn}\bm{\mathcal{A}}(\bm{X})\big{)}\|_{F}\leq\varepsilon^{\prime} for all $\bm{X}\in\widetilde{\mathcal{S}}$. This provides a $2\varepsilon$-covering of $\widetilde{\mathcal{S}}$ with $|\operatorname{sgn}\bm{\mathcal{A}}(\mathcal{S})|$ elements. Recall, however, that the latter is much smaller than the obvious bound $2^{m}$. Indeed, [4, Lemma 1] guarantees that the cardinality of the image under $\operatorname{sgn}\bm{\mathcal{A}}$ of a $d$-dimensional subspace $\mathcal{S}^{\prime}$ of $\mathbb{R}^{n\times n}$ satisfies $|\operatorname{sgn}\bm{\mathcal{A}}(\mathcal{S}^{\prime})|\leq(\frac{2m}{d})^{d}$. In our situation, we thus obtain a $2\varepsilon$-covering of $\widetilde{\mathcal{S}}$ with log-cardinality at most $rs\ln(2m/(rs))$. But, $\widetilde{\mathcal{S}}$ being isomorphic to the sphere $\mathbb{S}^{r\times s}_{F}$, the log-cardinality of such a covering must be at least $rs\ln(C/\varepsilon)$. Putting these two facts together gives $m\geq C\varepsilon^{-1}rs$, so that (4) is proved. ∎

II-B Upper estimate

Let us now establish the upper bound in (2). We first show that, with high probability on a Gaussian random map $\bm{\mathcal{A}}$, all pairs of matrices in a set $\mathcal{K}\subseteq\mathbb{B}^{n\times n}_{F}$ that are consistent under $\operatorname{sgn}\bm{\mathcal{A}}$ must be at most $\varepsilon$ apart provided $m$ is large compared to $\varepsilon$ and to the intrinsic set complexity. This complexity is measured by the Kolmogorov entropy of $\mathcal{K}$, i.e.,

[TABLE]

where the addition is the Minkowski sum between sets. Thus, the smallest number of translated Frobenius balls with radius $\eta$ that cover $\mathcal{K}$ is $\exp\mathcal{H}(\mathcal{K},\eta)$.

Theorem 2**.**

Let $\bm{\mathcal{A}}:\mathbb{R}^{n\times n}\to\mathbb{R}^{m}$ be a Gaussian random map. Given a subset $\mathcal{K}$ of $\mathbb{S}^{n\times n}_{F}$ and $\varepsilon>0$, if

[TABLE]

then it occurs with probability at least $1-\exp(-c\varepsilon m)$ that, for all ${\bm{X}},{\bm{X}}^{\prime}\in\mathcal{K}$,

[TABLE]

This result is related, but not strictly equivalent, to Theorem 2.4 from [5]. Under a requirement similar to (5), it states that a Gaussian random map $\bm{\mathcal{A}}$ defines, with high probability, a local $\varepsilon$-isometry of $\mathcal{K}$. In particular, it implies that, for all $\bm{X},\bm{X}^{\prime}\in\mathcal{K}$ such that $\|\bm{X}-\bm{X}^{\prime}\|_{F}\geq\varepsilon$, one has

[TABLE]

with $\|\bm{s},\bm{s}^{\prime}\|_{H}:=|\{i\in\llbracket m\rrbracket:s_{i}\neq s^{\prime}_{i}\}|$ for $\bm{s},\bm{s}^{\prime}\in\{\pm 1\}^{m}$. Note that (7) does not yield the contraposition of (6), since it does not enforce that $\operatorname{sgn}\bm{\mathcal{A}}\bm{X}\neq\operatorname{sgn}\bm{\mathcal{A}}\bm{X}^{\prime}$.

Proof of Theorem 2.

The stated version appears in [6, Theorem 1.5]. For the sake of diversity of the arguments, we follow the idea from [4, Theorem 2] and prove a result that would still imply the upper bound in (2), but does require strengthening the assumption (5) to

[TABLE]

We shall prove that $\mathbb{P}_{0}\leq\exp(-c\varepsilon m)$, where

[TABLE]

For a number $\rho\in(0,\varepsilon/2)$ yet to be chosen, we consider a $\rho$-covering $\{{\bm{X}}_{1},\,\cdots,{\bm{X}}_{K}\}$ of $\mathcal{K}$ with $K=\exp\mathcal{H}(\mathcal{K},\rho)$. Given ${\bm{X}},{\bm{X}}^{\prime}\in\mathcal{K}$ satisfying $\|{\bm{X}}-{\bm{X}}^{\prime}\|_{F}>\varepsilon$ and $\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}})=\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}}^{\prime})$, we select $k,k^{\prime}\in\llbracket K\rrbracket$ such that $\|{\bm{X}}-{\bm{X}}_{k}\|_{F}\leq\rho$ and $\|{\bm{X}}^{\prime}-{\bm{X}}_{k^{\prime}}\|_{F}\leq\rho$, implying that $\|{\bm{X}}_{k}-{\bm{X}}_{k^{\prime}}\|_{F}\geq\varepsilon-2\rho$ and that $\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}}_{k}+{\bm{Z}})=\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}}_{k^{\prime}}+{\bm{Z}}^{\prime})$ where ${\bm{Z}}:={\bm{X}}-{\bm{X}}_{k}$ and ${\bm{Z}}^{\prime}:={\bm{X}}^{\prime}-{\bm{X}}_{K^{\prime}}$ both have norm at most $\rho$. It follows that

[TABLE]

With $\Omega:=\{(k,k^{\prime})\in\llbracket K\rrbracket^{2}:\|{\bm{X}}_{k}-{\bm{X}}_{k^{\prime}}\|_{F}\geq\varepsilon-2\rho\}$, a union bound then gives

[TABLE]

where the summands are

[TABLE]

Let us observe that, for $(k,k^{\prime})\in\Omega$,

[TABLE]

where the last step was an application of [4, Lemma 9]. In view of $\arccos(\langle{\bm{X}}_{k},{\bm{X}}_{k^{\prime}}\rangle_{F})\geq\|{\bm{X}}_{k}-{\bm{X}}_{k^{\prime}}\|_{F}\geq\varepsilon-2\rho$ and of the choice $\rho=c\varepsilon/n$ for an appropriate $c$, we obtain

[TABLE]

Altogether, we conclude that

[TABLE]

which readily implies the result under the assumption (8). ∎

Interestingly, an explicit bound for the Kolmogorov entropy of the set of low-rank and bisparse matrices can be obtained as follows.

Proposition 2**.**

Given $r,s\in\llbracket n\rrbracket$ with $r\leq s$ and $\eta>0$, one has

[TABLE]

Proof.

Fixing subsets $S,T$ of $\llbracket n\rrbracket$ with $|S|=|T|=s$, one can find an $\eta$-covering $\mathcal{G}_{S,T}$ of $\widetilde{\Sigma}_{S\times T}^{[r]}$ with cardinality $|\mathcal{G}_{S,T}|\leq(9/\eta)^{r(2s+1)}$, see [7, Lemma 3.1]. In view of $\widetilde{\Sigma}_{(s)}^{[r]}\ =\ \bigcup_{S,T}\widetilde{\Sigma}_{S}^{[r]}$, an $\eta$-covering of $\widetilde{\Sigma}_{(s)}^{[r]}$ is then given by $\textstyle\mathcal{G}:=\bigcup_{S,T}\mathcal{G}_{S,T}$. Using Stirling’s bound, we derive that \mathcal{H}(\widetilde{\Sigma}_{(s)}^{[r]},\eta)\leq\ln|\mathcal{G}|\leq\ln\big{[}\binom{n}{s}^{2}(9/\eta)^{r(2s+1)}\big{]}\leq 2s\ln(en/s)+r(2s+1)\ln(9/\eta), which concludes the proof. ∎

Theorem 2 and Proposition 2 combine into the following result.

Corollary 1**.**

Let $\bm{\mathcal{A}}:\mathbb{R}^{n\times n}\to\mathbb{R}^{m}$ be a Gaussian random map. Given $\varepsilon>0$, if

[TABLE]

then it occurs with probability at least $1-\exp(-c\varepsilon m)$ that (6) is fulfilled for all ${\bm{X}},{\bm{X}}^{\prime}\in\mathcal{K}$.

With Corollary 1 at hand, we can justify the upper bound in (2). For this purpose, consider $\varepsilon>0$ making (9) an equality. The assumption of Theorem 2 then holds, so there is a linear map $\bm{\mathcal{A}}:\mathbb{R}^{n\times n}\to\mathbb{R}^{m}$ such that

[TABLE]

We conclude by remarking that

[TABLE]

which implies the desired upper bound on $\operatorname{CW}^{m}(\widetilde{\Sigma}_{(s)}^{[r]})$.

III Idealized algorithm for Gaussian sensing

Corollary 1 — as well as Theorem 2 for more general sets — says that any ${\bm{X}}\in\widetilde{\Sigma}_{(s)}^{[r]}$ is well approximated by any other ${\bm{X}}^{\prime}\in\widetilde{\Sigma}_{(s)}^{[r]}$ sharing the same one-bit measurements ${\bm{y}}\in\{\pm 1\}^{m}$. It does not, however, exhibit a scheme to create an approximant from ${\bm{y}}$. We now provide such a scheme by considering the projected back projection

[TABLE]

Here, $\mathcal{P}_{{\Sigma}_{(s)}^{[r]}}$ denotes the projection onto the set ${\Sigma}_{(s)}^{[r]}$, but we stress from the onset that computing it is not a realistic task, see [2] for a related NP-hardness statement. We note also that ${\bm{X}}^{\prime}$ need not be consistent, in the sense that $\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}}^{\prime})$ is not necessarily equal to ${\bm{y}}$, but that it has the same structure as ${\bm{X}}$. The essential tool behind our argument is a restricted isometry property for the set ${\Sigma}_{(s)}^{[r]}$, established below for Gaussian measurements.

Theorem 3**.**

Let $\bm{\mathcal{A}}:\mathbb{R}^{n\times n}\to\mathbb{R}^{m}$ be a Gaussian random map. Given $\delta\in(0,1)$, if

[TABLE]

then it occurs with probability at least $1-2\exp(-c\delta^{2}m)$ that, for all ${\bm{Z}}\in{\Sigma}_{(s)}^{[r]}$,

[TABLE]

Proof.

If we overlook the exact powers of $\delta$, the classical proof consisting of a concentration inequality followed by a covering argument will go through — the key ingredient being the estimate of [7, Lemma 3.1] for the covering number of the ball of ${\Sigma}_{S\times T}^{[r]}$ combined with a union bound over all index sets $S,T$ of size $s$.

Alternatively, in terms of Gaussian width, we can invoke the result from [8] (see also [9]) which guarantees that (12) holds provided

[TABLE]

Then, in view of $w(\cup_{k=1}^{K}\mathcal{K}_{k})\leq\max_{k}w(\mathcal{K}_{k})+3\sqrt{\ln K}$ for all subsets $\mathcal{K}_{1},\ldots,\mathcal{K}_{K}$ of the sphere (see [10, Lemma 12(ii)]) and of the fact that $\widetilde{\Sigma}_{(s)}^{[r]}=\cup_{S,T}\widetilde{\Sigma}_{S\times T}^{[r]}$ with $w(\widetilde{\Sigma}_{S\times T}^{[r]})^{2}\leq Crs$, we deduce

[TABLE]

It is clear that (13) holds under the assumption (11). ∎

We now state and prove the main result of this section. The argument is an adaptation of the technique presented in [11, Section 8.4].

Theorem 4**.**

Assuming that (12) holds on $\widetilde{\Sigma}_{(2s)}^{[2r]}$, any ${\bm{X}}\in\widetilde{\Sigma}_{(s)}^{[r]}$ acquired via ${\bm{y}}=\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}})$ is approximated by the matrix ${\bm{X}}^{\prime}$ defined in (10) with error

[TABLE]

Proof.

We write the singular value decompositions of ${\bm{X}}$ and ${\bm{X}}^{\prime}$ as $\textstyle{\bm{X}}=\sum_{k=1}^{r}\sigma_{k}{\bm{u}}_{k}{\bm{v}}_{k}^{\top}$ and ${\bm{X}}^{\prime}=\sum_{k=1}^{r}\sigma_{k}{\bm{u}}^{\prime}_{k}{\bm{v}}_{k}^{\prime\top}$, with the vectors ${\bm{u}}_{k},{\bm{v}}_{k}$ being supported on some $S,T$, $|S|=|T|=s$, and the vectors ${\bm{u}}^{\prime}_{k},{\bm{v}}^{\prime}_{k}$ being supported on some $S^{\prime},T^{\prime}$, $|S^{\prime}|=|T^{\prime}|=s$. Let also $\mathcal{P}_{\mathcal{S}}$ denote the orthogonal projection onto the linear space $\mathcal{S}={\rm span}\left\{{\bm{u}}_{1}{\bm{v}}_{1}^{\top},\ldots,{\bm{u}}_{r}{\bm{v}}_{r}^{\top},{\bm{u}}^{\prime}_{1}{\bm{v}}_{1}^{\prime\top},\ldots,{\bm{u}}^{\prime}_{r}{\bm{v}}_{r}^{\prime\top}\right\}\subseteq{\Sigma}_{(2s)}^{[2r]}$. Notice that $\mathcal{P}_{\mathcal{S}}({\bm{X}})={\bm{X}}$ and $\mathcal{P}_{\mathcal{S}}({\bm{X}}-{\bm{X}}^{\prime})={\bm{X}}-{\bm{X}}^{\prime}$ since ${\bm{X}}\in\mathcal{S}$ and ${\bm{X}}-{\bm{X}}^{\prime}\in\mathcal{S}$. Because the matrix ${\bm{X}}^{\prime}$ is the best approximant to $\bm{\mathcal{A}}^{*}\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}})$ from ${\Sigma}_{(s)}^{[r]}$, it is a better approximant to $\bm{\mathcal{A}}^{*}\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}})$ than ${\bm{X}}$ is, i.e., $\|\bm{\mathcal{A}}^{*}\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}})-{\bm{X}}^{\prime}\|_{F}^{2}\leq\|\bm{\mathcal{A}}^{*}\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}})-{\bm{X}}\|_{F}^{2}$. Introducing ${\bm{X}}$ in the left-hand side and expanding the square yields

[TABLE]

which implies that

[TABLE]

Taking $\|{\bm{X}}\|_{F}^{2}=1$ into account, we remark that

[TABLE]

Now, thanks to $\mathcal{P}_{\mathcal{S}}(\bm{\mathcal{A}}^{*}\operatorname{sgn}\bm{\mathcal{A}}({\bm{X}}))\in{\Sigma}_{(2s)}^{[2r]}$, we have

[TABLE]

which simplifies to

[TABLE]

Besides, thanks to $\mathcal{P}_{\mathcal{S}}({\bm{X}})={\bm{X}}$ and ${\bm{X}}\in{\Sigma}_{(s)}^{[r]}$, we also have

[TABLE]

Putting (17) and (18) together in (16) gives

[TABLE]

Finally, substituting in (15), we arrive at $\|{\bm{X}}-{\bm{X}}^{\prime}\|_{F}\leq C\sqrt{\delta}$. ∎

To close this section, we reformulate (14) by stating that, with high probability, the recovery error for projected back projection with Gaussian random maps achieves the decay rate

[TABLE]

since the restricted isometry property (12) holds in the regime (11).

IV Factorized Gaussian sensing

While the projected back projection scheme (10) is impractical, changing the sensing map $\bm{\mathcal{A}}:\mathbb{R}^{n\times n}\to\mathbb{R}^{m}$ allows for the design of a practical algorithm to recover matrices in $\widetilde{\Sigma}_{(s)}^{[r]}$ with provable recovery error decaying as a root of $rs\ln(n/s)/m$. The strategy follows a multistep argument put forward in [12] for sparse phase retrieval and also imitated in [2] for low-rank and bisparse recovery from standard, i.e., not quantized, measurements. It consists in defining a sensing map $\bm{\mathcal{A}}:\mathbb{R}^{n\times n}\to\mathbb{R}^{m}$ associated with matrices

[TABLE]

where $\bm{A}^{\prime}_{1},\ldots,\bm{A}^{\prime}_{m}\in\mathbb{R}^{p\times p}$ and $\bm{B},\bm{C}\in\mathbb{R}^{p\times n}$ for some $p$ chosen later. The one-bit measurements made on $\bm{X}\in\mathbb{R}^{n\times n}$ through the map $\bm{\mathcal{A}}:\mathbb{R}^{n\times n}\to\mathbb{R}^{m}$ can also be interpreted as one-bit measurements made on $\bm{B}\bm{X}\bm{C}^{\top}\in\mathbb{R}^{p\times p}$ through the map $\bm{\mathcal{A}}^{\prime}:\mathbb{R}^{p\times p}\to\mathbb{R}^{m}$ associated with the $\bm{A}^{\prime}_{i}$. Namely, in view of $\langle\bm{B}^{\top}\bm{A}^{\prime}_{i}\bm{C},\bm{X}\rangle_{F}=\langle\bm{A}^{\prime}_{i},\bm{B}\bm{X}\bm{C}^{\top}\rangle_{F}$, we have

[TABLE]

We will see that, under appropriate assumptions on $\bm{\mathcal{A}}^{\prime}$, $\bm{B}$, $\bm{C}$, a good approximant to $\bm{X}\in\widetilde{\Sigma}_{(s)}^{[r]}$ constructed from $\bm{y}=\operatorname{sgn}\bm{\mathcal{A}}(\bm{X})$ is

[TABLE]

Here, $\mathcal{H}^{[r]}$ denotes the operator of best rank-$r$ approximation (obtained by keeping $r$ leading singular elements), while $\mathcal{H}_{(s)}^{\rm row}$ (respectively $\mathcal{H}_{(s)}^{\rm col}$) stands for the operator of best approximation by $s$-row-sparse (respectively $s$-column-sparse) matrices, i.e., obtained by keeping $s$ rows (respectively $s$ columns) with largest $\ell_{2}$-norms. The appropriate assumption on $\bm{\mathcal{A}}^{\prime}$ is a restricted isometry property of order $2r$ with constant $\delta^{\prime}\in(0,1)$, reading

[TABLE]

whenever ${\bm{Z}}\in\mathbb{R}^{p\times p}$ has rank $\leq 2r$. The appropriate assumptions on ${\bm{D}}={\bm{B}},{\bm{C}}$ are standard restricted isometry properties of order $2s$ with constant $\delta\in(0,1)$, reading

[TABLE]

whenever ${\bm{z}}\in\mathbb{R}^{n}$ is $2s$-sparse. It is routine to verify that the later condition guarantees that, for any $k\geq 1$,

[TABLE]

whenever the row-supports of ${\bm{Z}},{\bm{Z}}^{\prime}\in\mathbb{R}^{n\times k}$ have combined size at most $2s$. Under this condition, for any $s$-row-sparse ${\bm{Z}}\in\mathbb{R}^{n\times k}$ seen through the inexact measurements ${\bm{Y}}={\bm{D}}{\bm{Z}}+{\bm{E}}\in\mathbb{R}^{p\times k}$, one has

[TABLE]

Writing ${\bm{Z}}^{\prime}=\mathcal{H}_{(s)}^{\rm row}[{\bm{D}}^{\top}{\bm{Y}}]$ and ${\bm{M}}={\bm{Z}}-{\bm{Z}}^{\prime}$, this comes from, using in turn the defining property of ${\bm{Z}}^{\prime}$, (23), and (22),

[TABLE]

We now state and prove the main result of this section.

Theorem 5**.**

Given $\delta,\delta^{\prime}\in(0,1)$, under assumptions (21) on $\bm{\mathcal{A}}^{\prime}$ and (22) on ${\bm{B}}$ and ${\bm{C}}$, any $\bm{X}\in\widetilde{\Sigma}_{(s)}^{[r]}$ acquired via ${\bm{y}}=\operatorname{sgn}\bm{\mathcal{A}}(\bm{X})$ is approximated by the matrix ${\bm{X}}^{\prime}$ given in (20) with error

[TABLE]

Proof.

In view of (19), the vector ${\bm{y}}\in\{\pm 1\}^{m}$ provides one-bit measurements made on the rank-$r$ matrix ${\bm{B}}{\bm{X}}{\bm{C}}^{\top}\in\mathbb{R}^{p\times p}$ through the sensing map $\bm{\mathcal{A}}^{\prime}:\mathbb{R}^{p\times p}\to\mathbb{R}^{m}$. Since this map satisfies the restricted isometry property (21), it is known (see [13, Theorem 2] or even Theorem 4 with $s=n=p$) that

[TABLE]

with $c_{X}:=\|{\bm{B}}{\bm{X}}{\bm{C}}^{\top}\|^{-1}_{F}$. As an aside, we notice that both matrices ${\bm{X}}{\bm{C}}^{\top}\in\mathbb{R}^{n\times p}$ and ${\bm{X}}^{\top}\in\mathbb{R}^{n\times n}$ are $s$-row-sparse, so that

[TABLE]

hence, since $\|{\bm{X}}{\bm{C}}^{\top}\|_{F}=\|{\bm{C}}{\bm{X}}^{\top}\|_{F}$ and $\|{\bm{X}}^{\top}\|_{F}=\|{\bm{X}}\|_{F}=1$,

[TABLE]

Now, viewing $\mathcal{H}^{[r]}(\bm{\mathcal{A}}^{\prime*}{\bm{y}})$ as inexact measurements made on the $s$-row-sparse matrix ${\bm{Z}}:=c_{X}{\bm{X}}{\bm{C}}^{\top}\in\mathbb{R}^{n\times p}$ via ${\bm{B}}$, namely $\mathcal{H}^{[r]}(\bm{\mathcal{A}}^{\prime*}{\bm{y}})={\bm{B}}{\bm{Z}}+{\bm{E}}$ with ${\bm{E}}:=\mathcal{H}^{[r]}(\bm{\mathcal{A}}^{\prime*}{\bm{y}})-c_{X}{\bm{B}}{\bm{X}}{\bm{C}}^{\top}$, (24) implies that

[TABLE]

where the last inequality used (26) and (25). Similarly, viewing (\mathcal{H}_{(s)}^{\rm row}\big{[}\,\bm{B}^{\top}\mathcal{H}^{[r]}(\bm{\mathcal{A}}^{\prime*}{\bm{y}})\,\big{]})^{\top}={\bm{C}}{\bm{Z}}^{\prime}+{\bm{E}}^{\prime} as inexact measurements made on the $s$-row-sparse matrix ${\bm{Z}}^{\prime}:=c_{X}{\bm{X}}^{\top}\in\mathbb{R}^{n\times n}$ via ${\bm{C}}$ with {\bm{E}}^{\prime}:=(\mathcal{H}_{(s)}^{\rm row}\big{[}\,\bm{B}^{\top}\mathcal{H}^{[r]}(\bm{\mathcal{A}}^{\prime*}{\bm{y}})\,\big{]})^{\top}-c_{X}{\bm{C}}{\bm{X}}^{\top}, (24) implies that

[TABLE]

where the last inequality used (28) and (29). By noticing that \mathcal{H}_{(s)}^{\rm row}({\bm{M}}^{\top})=\big{(}\mathcal{H}_{(s)}^{\rm col}({\bm{M}})\big{)}^{\top}, (30) actually reads

[TABLE]

The final result follows from a triangle inequality and the fact that $\|{\bm{X}}-c_{X}{\bm{X}}\|_{F}=|\|{\bm{B}}{\bm{X}}{\bm{C}}^{\top}\|_{F}-1|/\|{\bm{B}}{\bm{X}}{\bm{C}}^{\top}\|_{F}\leq C\delta$, which is a consequence of (28). ∎

We conclude by remarking that assumption (21) is valid when $\bm{\mathcal{A}}^{\prime}:\mathbb{R}^{p\times p}\to\mathbb{R}^{m}$ is a properly normalized Gaussian random map with $m\asymp\delta^{\prime-2}rp$ (see [13, Theorem 1] as an instantiation of [8]) and that assumption (22) is valid when ${\bm{B}},{\bm{C}}\in\mathbb{R}^{p\times n}$ are a properly normalized Gaussian random matrices with $p\asymp\delta^{-2}s\ln(n/s)$. Choosing $\delta^{\prime}=\delta^{2}$, we therefore arrive, with high probability, at $\|{\bm{X}}-{\bm{X}}^{\prime}\|_{F}\leq C\delta$ with $m\asymp\delta^{-6}rs\ln(n/s)$, i.e.,

[TABLE]

We note that the power $1/6$ is certainly not optimal — it can e.g. be increased by replacing $\mathcal{H}^{[r]}$ by a scheme improving on $\sqrt{\delta^{\prime}}$ in (25).

V Acknowledgments

S.F. is supported by NSF grants DMS-1622134 and DMS-1664803. L.J. is funded by the F.R.S.-FNRS. L.J. thanks L. Demanet for interesting discussions on factorized Gaussian sensing for sparse phase retrieval, e.g. on [12].