On Model Identification and Out-of-Sample Prediction of Principal Component Regression: Applications to Synthetic Controls

TL;DR

This paper provides new theoretical guarantees for principal component regression in high-dimensional settings, improving out-of-sample prediction accuracy and offering practical tools for model validation, with applications to synthetic controls.

Contribution

It introduces non-asymptotic prediction guarantees for PCR with fixed design, along with a linear algebraic condition for covariate shift, and extends results to synthetic control methods.

Findings

PCR consistently identifies the minimum $\

Non-asymptotic out-of-sample prediction bounds are established.

A hypothesis test for the key linear algebraic condition is proposed.

Abstract

We analyze principal component regression (PCR) in a high-dimensional error-in-variables setting with fixed design. Under suitable conditions, we show that PCR consistently identifies the unique model with minimum $\ell_2$-norm. These results enable us to establish non-asymptotic out-of-sample prediction guarantees that improve upon the best known rates. In the course of our analysis, we introduce a natural linear algebraic condition between the in- and out-of-sample covariates, which allows us to avoid distributional assumptions for out-of-sample predictions. Our simulations illustrate the importance of this condition for generalization, even under covariate shifts. Accordingly, we construct a hypothesis test to check when this conditions holds in practice. As a byproduct, our results also lead to novel results for the synthetic controls literature, a leading approach for policy evaluation. To the best of our knowledge, our prediction guarantees for the fixed design setting have been elusive in both the high-dimensional error-in-variables and synthetic controls literatures.