Convergence Rate of Krasulina Estimator

TL;DR

This paper analyzes the convergence rate of the Krasulina estimator for the least eigenvalue and eigenvector of a covariance matrix in PCA, providing a theoretical proof of convergence and its speed.

Contribution

It offers the first convergence proof and rate analysis for Krasulina's estimator in estimating the smallest eigenpair of a covariance matrix.

Findings

Proves convergence of Krasulina estimator for the least eigenvalue and eigenvector.

Derives the convergence rate of the estimator.

Provides theoretical guarantees for Krasulina's method in PCA.

Abstract

Principal component analysis (PCA) is one of the most commonly used statistical procedures with a wide range of applications. Consider the points $X_1, X_2,..., X_n$ are vectors drawn i.i.d. from a distribution with mean zero and covariance $\Sigma$, where $\Sigma$ is unknown. Let $A_n = X_nX_n^T$, then $E[A_n] = \Sigma$. This paper consider the problem of finding the least eigenvalue and eigenvector of matrix $\Sigma$. A classical such estimator are due to Krasulina\cite{krasulina_method_1969}. We are going to state the convergence proof of Krasulina for the least eigenvalue and corresponding eigenvector, and then find their convergence rate.