Accelerating Power Method with Fast Sketching for Stronger Low-Rank Approximation

TL;DR

This paper introduces a framework that accelerates the power method for low-rank matrix approximation by leveraging fast sketching techniques, improving efficiency and performance in large-scale data analysis.

Contribution

It develops a novel theoretical framework using regularized spectral approximation to enhance the power method with fast sketching, applicable to SVD and Nyström methods.

Findings

Achieves strong numerical performance on benchmark problems.

Provides provably efficient algorithms for low-rank approximation.

Introduces a flexible analysis framework using regularized spectral approximation.

Abstract

The power method is one of the most fundamental tools for extracting top principal components from data through low-rank matrix approximation. Yet, when the target rank is large, the cost of matrix multiplication associated with this procedure becomes a major bottleneck. We develop an algorithmic and theoretical framework for accelerating the power method using fast sketching, which is a popular paradigm in randomized linear algebra. Our framework leads to simple and provably efficient methods for singular value decomposition, low-rank factorization, and Nystr\"om approximation, which attain strong numerical performance on benchmark problems. The key novelty in our analysis is the use of regularized spectral approximation, a property of fast sketching methods which proves more flexible in generalizing power method guarantees than traditional arguments.