Empirical Bayes, SURE and Sparse Normal Mean Models

TL;DR

This paper explores empirical Bayes methods for sparse normal mean models, introducing mixture priors and a semiparametric estimator, with theoretical analysis and extensive numerical validation.

Contribution

It develops new empirical Bayes procedures with mixture priors and a semiparametric estimator, extending to heteroscedastic models and demonstrating their effectiveness.

Findings

Posterior median acts as a thresholding rule with multi-direction shrinkage.

Finite mixture priors effectively model cluster structures.

Proposed methods perform well in numerical studies.

Abstract

This paper studies the sparse normal mean models under the empirical Bayes framework. We focus on the mixture priors with an atom at zero and a density component centered at a data driven location determined by maximizing the marginal likelihood or minimizing the Stein Unbiased Risk Estimate. We study the properties of the corresponding posterior median and posterior mean. In particular, the posterior median is a thresholding rule and enjoys the multi-direction shrinkage property that shrinks the observation toward either the origin or the data-driven location. The idea is extended by considering a finite mixture prior, which is flexible to model the cluster structure of the unknown means. We further generalize the results to heteroscedastic normal mean models. Specifically, we propose a semiparametric estimator which can be calculated efficiently by combining the familiar EM algorithm with the Pool-Adjacent-Violators algorithm for isotonic regression. The effectiveness of our methods is demonstrated via extensive numerical studies.