Fast One-Pass Sparse Approximation of the Top Eigenvectors of Huge Approximately Low-Rank Matrices? Yes, $MAM^*$!

TL;DR

This paper introduces a provably-accurate, one-pass, compressive sensing-based algorithm for approximating top eigenvectors of massive low-rank matrices using minimal memory and computational resources.

Contribution

The paper presents a novel one-pass algorithm leveraging compressive sensing for sparse eigenvector approximation of large matrices, with theoretical guarantees and practical efficiency.

Findings

Effective approximation of eigenvectors for matrices with ~10^16 entries.

Memory footprint is comparable to the size of the sparse eigenvector.

Runtime depends mainly on the sparsity level, enabling sublinear complexity.

Abstract

Motivated by applications such as sparse PCA, in this paper we present provably-accurate one-pass algorithms for the sparse approximation of the top eigenvectors of extremely massive matrices based on a single compact linear sketch. The resulting compressive-sensing-based approaches can approximate the leading eigenvectors of huge approximately low-rank matrices that are too large to store in memory based on a single pass over its entries while utilizing a total memory footprint on the order of the much smaller desired sparse eigenvector approximations. Finally, the compressive sensing recovery algorithm itself (which takes the gathered compressive matrix measurements as input, and then outputs sparse approximations of its top eigenvectors) can also be formulated to run in a time which principally depends on the size of the sought sparse approximations, making its runtime sublinear in the size of the large matrix whose eigenvectors one aims to approximate. Preliminary experiments on huge matrices having $\sim 10^{16}$ entries illustrate the developed theory and demonstrate the practical potential of the proposed approach.