Improving Sketching Algorithms for Low-Rank Matrix Approximation via Sketch-Power Iterations

TL;DR

This paper introduces a single-pass, sketch-power iteration method that enhances low-rank matrix approximation accuracy in streaming settings without multiple data passes.

Contribution

It proposes a lightweight sketch-power iteration integrated into one-pass algorithms, reducing storage costs and improving approximation accuracy with theoretical error bounds.

Findings

Applying one or two sketch-power iterations significantly improves approximation accuracy.

The method effectively approximates dominant singular vectors on real-world data.

Error bounds guide parameter selection, matching observed optimal performance.

Abstract

Power iteration can improve the accuracy of randomized SVD, but requires multiple data passes, making it impractical in streaming or memory-constrained settings. We introduce a lightweight yet effective sketch-power iteration, allowing power-like iterations with only a single pass of the data, which can be incorporated into one-pass algorithms for low-rank approximation. As an example, we integrate the sketch-power iteration into a one-pass algorithm proposed by Tropp et al., and introduce strategies to reduce its storage cost. We establish meaningful error bounds: given a fixed storage budget, the sketch sizes derived from the bounds closely match the optimal ones observed in reality. This allows one to preselect reasonable parameters. Numerical experiments on both synthetic and real-world datasets indicate that, under the same storage constraints, applying one or two sketch-power iterations can substantially improve the approximation accuracy of the considered one-pass algorithms. In particular, experiments on real data with flat spectrum show that the method can approximate the dominant singular vectors well.