A Hierarchical Singular Value Decomposition Algorithm for Low Rank Matrices

TL;DR

This paper introduces a hierarchical merge-and-truncate algorithm for efficiently computing approximate truncated SVDs of low-rank matrices of arbitrary size, offering significant speedups with minimal error.

Contribution

It presents a novel tree-based method that merges block-wise truncated SVDs, applicable to matrices of any shape, improving efficiency over existing techniques.

Findings

Achieves over 3x speedup compared to traditional methods.

Maintains approximation error typically below 3%.

Applicable to dense, low-rank matrices in various applications.

Abstract

Singular value decomposition (SVD) is a widely used technique for dimensionality reduction and computation of basis vectors. In many applications, especially in fluid mechanics and image processing the matrices are dense, but low-rank matrices. In these cases, a truncated SVD corresponding to the most significant singular values is sufficient. In this paper, we propose a tree based merge-and-truncate algorithm to obtain an approximate truncated SVD of the matrix. Unlike previous methods, our technique is not limited to "tall and skinny" or "short and fat" matrices and it can be used for matrices of arbitrary size. The matrix is partitioned into blocks and the truncated SVDs of blocks are merged to obtain the final SVD. If the matrices are low rank, this algorithm gives significant speedup over finding the truncated SVD, even when run on a single core. The error is typically less than 3\%.