A Power Method for Computing Singular Value Decomposition

TL;DR

This paper introduces a power method for efficiently computing the top singular vectors of large matrices, useful for applications like PCA and low-rank approximation, with an implementation available in R.

Contribution

It proposes a novel power method for partial SVD computation that leverages neural network encoders and provides an accessible R package.

Findings

Effective for large matrices with fewer computations

The method's accuracy depends on singular value behavior and optimizer settings

Available implementation facilitates practical use

Abstract

The singular value decomposition (SVD) allows to write a matrix as a product of a left singular vectors matrix, a nonnegative singular values diagonal matrix and a right singular vectors matrix. Among the applications of the SVD are the principal component analysis, the low-rank matrix approximation and the solving of a linear system of equations. The methods used for computing this decomposition allow to get the complete or partial result. For very large size matrix, the probabilistic methods allow to get partial result by using less computational load. A power method is proposed in this paper for computing all or the $k$ first largest SVD subspaces for a real-valued matrix. The $k$ first right singular vectors of this method are the $k$ columns of a neural network encoder weight matrix. The accuracy of this iterative search method depends on the behavior of the singular values and the settings of the gradient search optimizer used. A R package implementing the proposed method is available at https://cran.r-project.org/web/packages/psvd/index.html.