What is a Sketch-and-Precondition Derivation for Low-Rank Approximation? Inverse Power Error or Inverse Power Estimation?

TL;DR

This paper introduces EPSI, a novel sketch-and-precondition method for randomized low-rank matrix approximation, providing theoretical guarantees and improved convergence rates that scale with sketch size.

Contribution

The paper develops the Error-Powered Sketched Inverse Iteration (EPSI) method, extending sketch-and-precondition techniques to low-rank approximation with proven convergence guarantees.

Findings

Convergence rate improves linearly with sketch size.

EPSI achieves theoretical guarantees for low-rank approximation.

Method accelerates iterative solvers while maintaining accuracy.

Abstract

Randomized sketching accelerates large-scale numerical linear algebra by reducing computational complexity. While the traditional sketch-and-solve approach reduces the problem size directly through sketching, the sketch-and-precondition method leverages sketching to construct a computational friendly preconditioner. This preconditioner improves the convergence speed of iterative solvers applied to the original problem, maintaining accuracy in the full space. Furthermore, the convergence rate of the solver improves at least linearly with the sketch size. Despite its potential, developing a sketch-and-precondition framework for randomized algorithms in low-rank matrix approximation remains an open challenge. We introduce the Error-Powered Sketched Inverse Iteration (EPSI) Method via run sketched Newton iteration for the Lagrange form as a sketch-and-precondition variant for randomized low-rank approximation. Our method achieves theoretical guarantees, including a convergence rate that improves at least linearly with the sketch size.