Faster Algorithms for Schatten-p Low Rank Approximation

TL;DR

This paper develops faster algorithms for Schatten-$p$ Low Rank Approximation by leveraging advanced matrix multiplication techniques and combining existing algorithms, while also providing stability analysis for practical implementation.

Contribution

It introduces improved algorithms for Schatten-$p$ LRA using fast matrix multiplication and combines them with existing methods, along with a stability analysis for floating point implementation.

Findings

Enhanced running times for Schatten-$p$ LRA algorithms.

First stability analysis for algorithms with polylogarithmic bits of precision.

Implementation of LazySVD with minimal precision requirements.

Abstract

We study algorithms for the Schatten-$p$ Low Rank Approximation (LRA) problem. First, we show that by using fast rectangular matrix multiplication algorithms and different block sizes, we can improve the running time of the algorithms in the recent work of Bakshi, Clarkson and Woodruff (STOC 2022). We then show that by carefully combining our new algorithm with the algorithm of Li and Woodruff (ICML 2020), we can obtain even faster algorithms for Schatten-$p$ LRA. While the block-based algorithms are fast in the real number model, we do not have a stability analysis which shows that the algorithms work when implemented on a machine with polylogarithmic bits of precision. We show that the LazySVD algorithm of Allen-Zhu and Li (NeurIPS 2016) can be implemented on a floating point machine with only logarithmic, in the input parameters, bits of precision. As far as we are aware, this is the first stability analysis of any algorithm using $O((k/\sqrt{\varepsilon})\text{poly}(\log n))$ matrix-vector products with the matrix $A$ to output a $1+\varepsilon$ approximate solution for the rank-$k$ Schatten-$p$ LRA problem.