ROIPCA: An online memory-restricted PCA algorithm based on rank-one updates

TL;DR

This paper introduces ROIPCA and fROIPCA, two online PCA algorithms based on rank-one updates, suitable for large-scale or streaming data, with proven theoretical properties and demonstrated empirical advantages.

Contribution

The paper presents two novel online PCA algorithms, ROIPCA and fROIPCA, with theoretical analysis and empirical validation showing improved accuracy and speed over existing methods.

Findings

fROIPCA is faster and has comparable accuracy to ROIPCA.

fROIPCA is shown to be a gradient algorithm with an optimal learning rate.

Numerical experiments demonstrate advantages over state-of-the-art algorithms.

Abstract

Principal components analysis (PCA) is a fundamental algorithm in data analysis. Its memory-restricted online versions are useful in many modern applications, where the data are too large to fit in memory, or when data arrive as a stream of items. In this paper, we propose ROIPCA and fROIPCA, two online PCA algorithms that are based on rank-one updates. While ROIPCA is typically more accurate, fROIPCA is faster and has comparable accuracy. We show the relation between fROIPCA and an existing popular gradient algorithm for online PCA, and in particular, prove that fROIPCA is in fact a gradient algorithm with an optimal learning rate. We demonstrate numerically the advantages of our algorithms over existing state-of-the-art algorithms in terms of accuracy and runtime.