SVD update methods for large matrices and applications

TL;DR

This paper introduces an efficient method for updating the singular value decomposition of large, tall-thin matrices when new data is added, facilitating applications in principal component analysis and kernel extraction.

Contribution

It presents a novel, efficient approach for updating the SVD of large matrices after augmentation, improving computational speed and storage for high-dimensional data analysis.

Findings

Method efficiently updates SVD for augmented matrices

Applicable to principal component analysis and kernel extraction

Demonstrated effectiveness in concrete applications

Abstract

We consider the problem of updating the SVD when augmenting a "tall thin" matrix, i.e., a rectangular matrix $A \in \RR^{m \times n}$ with $m \gg n$. Supposing that an SVD of $A$ is already known, and given a matrix $B \in \RR^{m \times n'}$, we derive an efficient method to compute and efficiently store the SVD of the augmented matrix $[ A B ] \in \RR^{m \times (n+n')}$. This is an important tool for two types of applications: in the context of principal component analysis, the dominant left singular vectors provided by this decomposition form an orthonormal basis for the best linear subspace of a given dimension, while from the right singular vectors one can extract an orthonormal basis of the kernel of the matrix. We also describe two concrete applications of these concepts which motivated the development of our method and to which it is very well adapted.