Shrinkage estimators for prediction out-of-sample: Conditional performance

TL;DR

This paper analyzes the out-of-sample predictive performance of shrinkage estimators like James-Stein in linear models, revealing they can perform worse than maximum-likelihood estimators when evaluated conditionally on fixed design variables.

Contribution

It provides a detailed conditional analysis of shrinkage estimators' out-of-sample prediction performance, contrasting with previous unconditional evaluations.

Findings

James-Stein estimator can underperform in out-of-sample prediction

Conditional evaluation reveals different performance dynamics

Contrasts with prior unconditional performance results

Abstract

We find that, in a linear model, the James-Stein estimator, which dominates the maximum-likelihood estimator in terms of its in-sample prediction error, can perform poorly compared to the maximum-likelihood estimator in out-of-sample prediction. We give a detailed analysis of this phenomenon and discuss its implications. When evaluating the predictive performance of estimators, we treat the regressor matrix in the training data as fixed, i.e., we condition on the design variables. Our findings contrast those obtained by Baranchik (1973, Ann. Stat. 1:312-321) and, more recently, by Dicker (2012, arXiv:1102.2952) in an unconditional performance evaluation.