Shrinkage Estimation in Multilevel Normal Models

TL;DR

This paper reviews the development of shrinkage estimators in multilevel normal models, highlighting the evolution from Stein's pioneering work to modern Bayesian approaches and their properties.

Contribution

It provides a comprehensive overview of the theoretical advancements in shrinkage estimation, including the characterization of admissible minimax estimators and the role of different priors.

Findings

Stein's estimator improves upon sample means for multiple normal populations.

The harmonic prior induces Stein's harmonic estimator with desirable properties.

Scale-invariance and conjugacy are key features of the unique flat prior in this context.

Abstract

This review traces the evolution of theory that started when Charles Stein in 1955 [In Proc. 3rd Berkeley Sympos. Math. Statist. Probab. I (1956) 197--206, Univ. California Press] showed that using each separate sample mean from $k\ge3$ Normal populations to estimate its own population mean $\mu_i$ can be improved upon uniformly for every possible $\mu=(\mu_1,...,\mu_k)'$. The dominating estimators, referred to here as being "Model-I minimax," can be found by shrinking the sample means toward any constant vector. Admissible minimax shrinkage estimators were derived by Stein and others as posterior means based on a random effects model, "Model-II" here, wherein the $\mu_i$ values have their own distributions. Section 2 centers on Figure 2, which organizes a wide class of priors on the unknown Level-II hyperparameters that have been proved to yield admissible Model-I minimax shrinkage estimators in the "equal variance case." Putting a flat prior on the Level-II variance is unique in this class for its scale-invariance and for its conjugacy, and it induces Stein's harmonic prior (SHP) on $\mu_i$.