The Fast Convergence of Incremental PCA
Akshay Balsubramani, Sanjoy Dasgupta, Yoav Freund

TL;DR
This paper analyzes the convergence rates of classical incremental PCA algorithms, Krasulina and Oja, providing finite-sample guarantees for estimating the top eigenvector in high-dimensional settings.
Contribution
It offers the first finite-sample convergence analysis for Krasulina and Oja algorithms in the context of incremental PCA.
Findings
Finite-sample convergence rates established for Krasulina and Oja algorithms.
Algorithms maintain O(d) space complexity while updating estimates.
Results demonstrate efficient top eigenvector estimation in high dimensions.
Abstract
We consider a situation in which we see samples in drawn i.i.d. from some distribution with mean zero and unknown covariance A. We wish to compute the top eigenvector of A in an incremental fashion - with an algorithm that maintains an estimate of the top eigenvector in O(d) space, and incrementally adjusts the estimate with each new data point that arrives. Two classical such schemes are due to Krasulina (1969) and Oja (1983). We give finite-sample convergence rates for both.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStatistical Methods and Inference · Advanced Statistical Methods and Models · Statistical Methods and Bayesian Inference
