Sketching for Simultaneously Sparse and Low-Rank Covariance Matrices

TL;DR

This paper presents a novel sketching technique for efficiently estimating structured covariance matrices that are both sparse and low-rank, using minimal observations and a specialized two-stage algorithm.

Contribution

It introduces a new sketching-based method for covariance estimation that leverages the matrix's structure and a two-stage algorithm for improved accuracy and efficiency.

Findings

Estimation accuracy is achieved with a number of sketches proportional to the maximum of rank and sparsity.

The algorithm exploits low-rank structure to operate on smaller matrices, reducing computational complexity.

The method is effective when sketching vectors have a specific structured form.

Abstract

We introduce a technique for estimating a structured covariance matrix from observations of a random vector which have been sketched. Each observed random vector $\boldsymbol{x}_t$ is reduced to a single number by taking its inner product against one of a number of pre-selected vector $\boldsymbol{a}_\ell$. These observations are used to form estimates of linear observations of the covariance matrix $\boldsymbol{\varSigma}$, which is assumed to be simultaneously sparse and low-rank. We show that if the sketching vectors $\boldsymbol{a}_\ell$ have a special structure, then we can use straightforward two-stage algorithm that exploits this structure. We show that the estimate is accurate when the number of sketches is proportional to the maximum of the rank times the number of significant rows/columns of $\boldsymbol{\varSigma}$. Moreover, our algorithm takes direct advantage of the low-rank structure of $\boldsymbol{\varSigma}$ by only manipulating matrices that are far smaller than the original covariance matrix.