On the Error-Propagation of Inexact Hotelling's Deflation for Principal Component Analysis

TL;DR

This paper analyzes how numerical errors propagate in inexact Hotelling's deflation method for PCA, providing explicit bounds and insights into error accumulation during sequential principal component extraction.

Contribution

It offers a mathematical characterization of error propagation in inexact Hotelling's deflation for PCA, including bounds for both generic sub-routines and power iteration.

Findings

Error propagation bounds are explicitly derived for inexact Hotelling's deflation.

Power iteration yields tighter error bounds compared to generic sub-routines.

The analysis informs the design of more accurate PCA algorithms using deflation.

Abstract

Principal Component Analysis (PCA) aims to find subspaces spanned by the so-called principal components that best represent the variance in the dataset. The deflation method is a popular meta-algorithm that sequentially finds individual principal components, starting from the most important ones and working towards the less important ones. However, as deflation proceeds, numerical errors from the imprecise estimation of principal components propagate due to its sequential nature. This paper mathematically characterizes the error propagation of the inexact Hotelling's deflation method. We consider two scenarios: $i)$ when the sub-routine for finding the leading eigenvector is abstract and can represent various algorithms; and $ii)$ when power iteration is used as the sub-routine. In the latter case, the additional directional information from power iteration allows us to obtain a tighter error bound than the sub-routine agnostic case. For both scenarios, we explicitly characterize how the errors progress and affect subsequent principal component estimations.