Bootstrapping the error of Oja's algorithm

TL;DR

This paper develops a bootstrap-based method to quantify the uncertainty in the estimation error of the leading eigenvector obtained via Oja's streaming PCA algorithm, combining advanced probabilistic tools.

Contribution

It introduces a weighted chi-squared approximation for the error and proposes an online multiplier bootstrap for uncertainty quantification in streaming PCA.

Findings

The bootstrap method provides consistent inference for the eigenvector error.

The approximation accurately captures the distribution of the estimation error.

Conditions for the bootstrap's validity are rigorously established.

Abstract

We consider the problem of quantifying uncertainty for the estimation error of the leading eigenvector from Oja's algorithm for streaming principal component analysis, where the data are generated IID from some unknown distribution. By combining classical tools from the U-statistics literature with recent results on high-dimensional central limit theorems for quadratic forms of random vectors and concentration of matrix products, we establish a weighted $\chi^2$ approximation result for the $\sin^2$ error between the population eigenvector and the output of Oja's algorithm. Since estimating the covariance matrix associated with the approximating distribution requires knowledge of unknown model parameters, we propose a multiplier bootstrap algorithm that may be updated in an online manner. We establish conditions under which the bootstrap distribution is close to the corresponding sampling distribution with high probability, thereby establishing the bootstrap as a consistent inferential method in an appropriate asymptotic regime.