Performance of Empirical Risk Minimization For Principal Component Regression

TL;DR

This paper provides nonparametric bounds on the predictive performance of empirical risk minimization in principal component regression, demonstrating its consistency and near-optimality across different signal regimes.

Contribution

It establishes theoretical performance bounds for ERM in PCR without assuming a specific data-generating model, covering both strong and weak signal scenarios.

Findings

ERM for PCR is consistent for prediction.

Achieves near-optimal performance in strong and weak signal regimes.

Provides nonparametric bounds independent of specific model assumptions.

Abstract

This paper establishes bounds on the predictive performance of empirical risk minimization for principal component regression. Our analysis is nonparametric, in the sense that the relation between the prediction target and the predictors is not specified. In particular, we do not rely on the assumption that the prediction target is generated by a factor model. In our analysis we consider the cases in which the largest eigenvalues of the covariance matrix of the predictors grow linearly in the number of predictors (strong signal regime) or sublinearly (weak signal regime). The main result of this paper shows that empirical risk minimization for principal component regression is consistent for prediction and, under appropriate conditions, it achieves near-optimal performance in both the strong and weak signal regimes.