Exact Recovery of Sparse Binary Vectors from Generalized Linear Measurements

TL;DR

This paper establishes the optimal sample complexity for exactly recovering sparse binary vectors from generalized linear measurements, including noisy one-bit and logistic regression, showing no statistical-computational gap exists for these problems.

Contribution

The paper provides tight bounds on the number of measurements needed for exact recovery in sparse binary vector problems, demonstrating computational efficiency and resolving conjectures about statistical-computational gaps.

Findings

Sample complexity of O((k+σ^2) log n) for noisy one-bit measurements.

Optimality of the sample complexity due to information-theoretic bounds.

No statistical-computational gap for binary compressed sensing and logistic regression.

Abstract

We consider the problem of exact recovery of a $k$-sparse binary vector from generalized linear measurements (such as logistic regression). We analyze the linear estimation algorithm (Plan, Vershynin, Yudovina, 2017), and also show information theoretic lower bounds on the number of required measurements. As a consequence of our results, for noisy one bit quantized linear measurements ($\mathsf{1bCSbinary}$), we obtain a sample complexity of $O((k+\sigma^2)\log{n})$, where $\sigma^2$ is the noise variance. This is shown to be optimal due to the information theoretic lower bound. We also obtain tight sample complexity characterization for logistic regression. Since $\mathsf{1bCSbinary}$ is a strictly harder problem than noisy linear measurements ($\mathsf{SparseLinearReg}$) because of added quantization, the same sample complexity is achievable for $\mathsf{SparseLinearReg}$. While this sample complexity can be obtained via the popular lasso algorithm, linear estimation is computationally more efficient. Our lower bound holds for any set of measurements for $\mathsf{SparseLinearReg}$, (similar bound was known for Gaussian measurement matrices) and is closely matched by the maximum-likelihood upper bound. For $\mathsf{SparseLinearReg}$, it was conjectured in Gamarnik and Zadik, 2017 that there is a statistical-computational gap and the number of measurements should be at least $(2k+\sigma^2)\log{n}$ for efficient algorithms to exist. It is worth noting that our results imply that there is no such statistical-computational gap for $\mathsf{1bCSbinary}$ and logistic regression.