A note on the variance in principal component regression

TL;DR

This paper discusses how principal component regression can lead to increased variance and lack of fit when key dimensions are omitted, highlighting limitations not evident from eigenvalues.

Contribution

It reveals that PC-regression can suffer from increased variance even when eigenvalues do not indicate issues, providing new insights into its limitations.

Findings

PC-regression can have higher variance than OLS when important dimensions are omitted.

Eigenvalues alone do not reveal the variance issues in PC-regression.

Omission of key dimensions affects the estimator's variance and fit.

Abstract

Principal component regression results in lack of fit when important dimensions are omitted, which cannot be assessed from the eigenvalues. I show that the PC-regression estimator can also suffer from increased variance relative to ordinary least squares in such cases.