Fixed-Rank Approximation of a Positive-Semidefinite Matrix from Streaming Data

TL;DR

This paper introduces a new algorithm for approximating large positive-semidefinite matrices from streaming data using sketches, combining Nystrom approximation with a novel rank truncation method, and demonstrates superior performance over existing techniques.

Contribution

The paper presents a novel fixed-rank PSD matrix approximation algorithm from streaming data that integrates Nystrom approximation with a new rank truncation mechanism, supported by theoretical guarantees.

Findings

Achieves prescribed relative error in Schatten 1-norm

Exploits spectral decay of input matrices

Outperforms existing methods in experiments

Abstract

Several important applications, such as streaming PCA and semidefinite programming, involve a large-scale positive-semidefinite (psd) matrix that is presented as a sequence of linear updates. Because of storage limitations, it may only be possible to retain a sketch of the psd matrix. This paper develops a new algorithm for fixed-rank psd approximation from a sketch. The approach combines the Nystrom approximation with a novel mechanism for rank truncation. Theoretical analysis establishes that the proposed method can achieve any prescribed relative error in the Schatten 1-norm and that it exploits the spectral decay of the input matrix. Computer experiments show that the proposed method dominates alternative techniques for fixed-rank psd matrix approximation across a wide range of examples.