An implementation of a randomized algorithm for principal component analysis

TL;DR

This paper introduces a robust MATLAB implementation of randomized algorithms for PCA and low-rank approximation, demonstrating superior efficiency and ease-of-use over classical methods in most scenarios.

Contribution

It provides an easy-to-use, black-box MATLAB implementation of randomized PCA algorithms, outperforming traditional techniques in accuracy, speed, memory, and parallelizability.

Findings

Randomized algorithms match or outperform classical methods in accuracy and efficiency.

The implementation is reliable, easy to use, and suitable for parallel computing.

Classical methods remain superior for spectral norm estimation and least singular value calculations.

Abstract

Recent years have witnessed intense development of randomized methods for low-rank approximation. These methods target principal component analysis (PCA) and the calculation of truncated singular value decompositions (SVD). The present paper presents an essentially black-box, fool-proof implementation for Mathworks' MATLAB, a popular software platform for numerical computation. As illustrated via several tests, the randomized algorithms for low-rank approximation outperform or at least match the classical techniques (such as Lanczos iterations) in basically all respects: accuracy, computational efficiency (both speed and memory usage), ease-of-use, parallelizability, and reliability. However, the classical procedures remain the methods of choice for estimating spectral norms, and are far superior for calculating the least singular values and corresponding singular vectors (or singular subspaces).