Projection techniques to update the truncated SVD of evolving matrices

TL;DR

This paper introduces a purely algebraic projection-based algorithm for efficiently updating the truncated SVD of matrices as they evolve with new data, improving accuracy especially for dominant singular values.

Contribution

It proposes a novel projection technique for updating the truncated SVD of evolving matrices, with analysis and practical insights for better accuracy.

Findings

Higher accuracy for largest singular values

Effective for matrices from real applications

Two different projection subspace choices analyzed

Abstract

This paper considers the problem of updating the rank-k truncated Singular Value Decomposition (SVD) of matrices subject to the addition of new rows and/or columns over time. Such matrix problems represent an important computational kernel in applications such as Latent Semantic Indexing and Recommender Systems. Nonetheless, the proposed framework is purely algebraic and targets general updating problems. The algorithm presented in this paper undertakes a projection view-point and focuses on building a pair of subspaces which approximate the linear span of the sought singular vectors of the updated matrix. We discuss and analyze two different choices to form the projection subspaces. Results on matrices from real applications suggest that the proposed algorithm can lead to higher accuracy, especially for the singular triplets associated with the largest modulus singular values. Several practical details and key differences with other approaches are also discussed.