Near-Optimal Estimation of Simultaneously Sparse and Low-Rank Matrices from Nested Linear Measurements

TL;DR

This paper introduces a nearly optimal two-stage algorithm for estimating matrices that are both low-rank and row-wise sparse from nested linear measurements, with theoretical guarantees and practical evaluations.

Contribution

It proposes a computationally efficient method with near-minimax optimal accuracy guarantees for simultaneously structured matrix estimation from nested measurements.

Findings

The algorithm achieves accuracy close to the minimax lower bound.

The method is computationally efficient and scalable.

Numerical simulations validate the theoretical results.

Abstract

In this paper we consider the problem of estimating simultaneously low-rank and row-wise sparse matrices from nested linear measurements where the linear operator consists of the product of a linear operator $\mathcal{W}$ and a matrix $\mathbf{\varPsi}$. Leveraging the nested structure of the measurement operator, we propose a computationally efficient two-stage algorithm for estimating the simultaneously structured target matrix. Assuming that $\mathcal{W}$ is a restricted isometry for low-rank matrices and $\mathbf{\varPsi}$ is a restricted isometry for row-wise sparse matrices, we establish an accuracy guarantee that holds uniformly for all sufficiently low-rank and row-wise sparse matrices with high probability. Furthermore, using standard tools from information theory, we establish a minimax lower bound for estimation of simultaneously low-rank and row-wise sparse matrices from linear measurements that need not be nested. The accuracy bounds established for the algorithm, that also serve as a minimax upper bound, differ from the derived minimax lower bound merely by a polylogarithmic factor of the dimensions. Therefore, the proposed algorithm is nearly minimax optimal. We also discuss some applications of the proposed observation model and evaluate our algorithm through numerical simulation.