Empirical Bayes Estimation and Inference via Smooth Nonparametric Maximum Likelihood

TL;DR

This paper introduces a smooth NPMLE approach for empirical Bayes estimation in normal means problems, improving uncertainty quantification and convergence rates over traditional methods.

Contribution

It proposes a hierarchical Gaussian smoothing layer to produce a smooth NPMLE that achieves near-parametric denoising and polynomial convergence rates, with theoretical guarantees.

Findings

Smooth NPMLE inherits classical denoising performance.

Achieves polynomial convergence rates for deconvolution.

Constructs marginal coverage sets with asymptotic exactness.

Abstract

The empirical Bayes $g$-modeling approach via the nonparametric maximum likelihood estimator (NPMLE) is widely used for large-scale estimation and inference in the normal means problem, yet theoretical guarantees for uncertainty quantification remain scarce. A key obstacle is that the NPMLE of the mixing distribution is necessarily discrete, which yields discrete posterior credible sets and a deconvolution rate that is logarithmic. We address both limitations by studying a hierarchical Gaussian smoothing layer that restricts the mixing distribution to a Gaussian location mixture. The resulting smooth NPMLE is computed by solving a convex optimization problem and inherits the near-parametric denoising performance of the classical NPMLE. For deconvolution it achieves a polynomial rate of convergence which we show is asymptotically minimax over the corresponding class. The estimated smooth posteriors converge to the true posteriors at the same polynomial rate in weighted total variation distance. When the model is misspecified, the smooth NPMLE converges to the Kullback-Leibler projection of the true marginal density onto the model class at a nearly parametric rate, and the polynomial deconvolution and posterior convergence rates carry over to this pseudo-true target. Building on this smooth posterior, we characterize optimal marginal coverage sets: the shortest set-valued rules achieving a prescribed marginal coverage probability. Plug-in empirical Bayes marginal coverage sets based on the smooth NPMLE achieve asymptotically exact coverage at a polynomial rate and converge to the oracle optimal set in expected length. All results extend to heteroscedastic Gaussian observations. We also study identifiability of the proposed model and show that the largest Gaussian component of the prior is identifiable, and provide a consistent estimator and a finite-sample upper confidence bound for it.