Accurate and fast matrix factorization for low-rank learning

TL;DR

This paper introduces two novel methods based on Krylov subspaces for fast and accurate partial singular value decomposition and rank estimation of large matrices, outperforming traditional approaches.

Contribution

It presents new algorithms leveraging Krylov subspaces for efficient low-rank matrix computations, suitable for large-scale data applications.

Findings

Methods outperform traditional SVD in accuracy and speed

Effective on large matrices like MNIST and USPS datasets

Suitable for applications requiring precise singular vectors

Abstract

In this paper, we tackle two important problems in low-rank learning, which are partial singular value decomposition and numerical rank estimation of huge matrices. By using the concepts of Krylov subspaces such as Golub-Kahan bidiagonalization (GK-bidiagonalization) as well as Ritz vectors, we propose two methods for solving these problems in a fast and accurate way. Our experiments show the advantages of the proposed methods compared to the traditional and randomized singular value decomposition methods. The proposed methods are appropriate for applications involving huge matrices where the accuracy of the desired singular values and also all of their corresponding singular vectors are essential. As a real application, we evaluate the performance of our methods on the problem of Riemannian similarity learning between two various image datasets of MNIST and USPS.