Beyond Sin-Squared Error: Linear-Time Entrywise Uncertainty Quantification for Streaming PCA

TL;DR

This paper introduces a new statistical inference framework for streaming PCA using Oja's algorithm, enabling entrywise uncertainty quantification with sharp error bounds and efficient variance estimation methods.

Contribution

It provides the first sharp entrywise uncertainty quantification for streaming PCA, including a Bernstein-type concentration bound, a CLT, and a computationally efficient variance estimation algorithm.

Findings

Derived a sharp Bernstein-type concentration bound for eigenvector entries.

Established a CLT for a subset of eigenvector entries.

Proposed a median-of-means based variance estimator with empirical accuracy.

Abstract

We propose a novel statistical inference framework for streaming principal component analysis (PCA) using Oja's algorithm, enabling the construction of confidence intervals for individual entries of the estimated eigenvector. Most existing works on streaming PCA focus on providing sharp sin-squared error guarantees. Recently, there has been some interest in uncertainty quantification for the sin-squared error. However, uncertainty quantification or sharp error guarantees for entries of the estimated eigenvector in the streaming setting remains largely unexplored. We derive a sharp Bernstein-type concentration bound for elements of the estimated vector matching the optimal error rate up to logarithmic factors. We also establish a Central Limit Theorem for a suitably centered and scaled subset of the entries. To efficiently estimate the coordinate-wise variance, we introduce a provably consistent subsampling algorithm that leverages the median-of-means approach, empirically achieving similar accuracy to multiplier bootstrap methods while being significantly more computationally efficient. Numerical experiments demonstrate its effectiveness in providing reliable uncertainty estimates with a fraction of the computational cost of existing methods.