Tutorial: Complexity analysis of Singular Value Decomposition and its variants

TL;DR

This paper compares various SVD methods, including regular, truncated, Krylov, and randomized PCA, analyzing their time and space complexities, especially when computing all eigenpairs, and discusses their relation to PCA.

Contribution

It provides a comparative analysis of SVD variants and their efficiency for computing all eigenpairs, highlighting their suitability based on problem size.

Findings

Krylov and randomized PCA perform well only when k << n

Comparison of time and space complexity for different SVD methods

Discussion of the relationship between PCA and SVD

Abstract

We compared the regular Singular Value Decomposition (SVD), truncated SVD, Krylov method and Randomized PCA, in terms of time and space complexity. It is well-known that Krylov method and Randomized PCA only performs well when k << n, i.e. the number of eigenpair needed is far less than that of matrix size. We compared them for calculating all the eigenpairs. We also discussed the relationship between Principal Component Analysis and SVD.