Average performance analysis of the stochastic gradient method for online PCA

TL;DR

This paper analyzes the efficiency of stochastic gradient methods for online PCA, introduces an adaptive learning rate strategy, and demonstrates through simulations that this approach significantly improves performance.

Contribution

It provides a complexity analysis of stochastic gradient PCA in streaming data and proposes an online learning rate selection method for enhanced results.

Findings

Stochastic gradient PCA has quantifiable complexity in streaming settings.

Adaptive learning rate improves convergence and performance.

Simulations confirm practical benefits of the proposed method.

Abstract

This paper studies the complexity of the stochastic gradient algorithm for PCA when the data are observed in a streaming setting. We also propose an online approach for selecting the learning rate. Simulation experiments confirm the practical relevance of the plain stochastic gradient approach and that drastic improvements can be achieved by learning the learning rate.