Function estimation in the empirical Bayes setting

TL;DR

This paper investigates the estimation of smooth functions of latent parameters in empirical Bayes models, revealing faster convergence rates for polynomial functions and establishing a hierarchy of difficulty in empirical Bayes problems.

Contribution

The paper provides tight bounds for estimating polynomial functions in empirical Bayes, linking approximation theory with statistical inverse problems, and highlights the varying difficulty levels in empirical Bayes estimation.

Findings

Polynomial functions can be estimated at near-parametric rates.

Lipschitz functions require denser polynomial approximations, incurring higher error.

Sharp hierarchy of empirical Bayes problem difficulty is established.

Abstract

We study function estimation in the empirical Bayes setting for Poisson and normal means. Specifically, given observations $Y_i\sim f(\cdot; \theta_i)$ with latent parameters $\theta_i\sim \pi$, the goal is to estimate $\mathbb{E}_{\pi}[\ell(\theta)|X = x]$. This task lies between classical deconvolution (recovering the full prior $\pi$), and standard empirical Bayes mean estimation. While the minimax risk for estimating $\pi$ in the Wasserstein distance is known to decay only logarithmically, we show that estimating certain smooth functions admits dramatically faster rates. In particular, for polynomial functions of degree $k$ in the Poisson model, we establish a tight bound of $\Theta(\frac{1}{n}(\frac{\log n}{\log \log n})^{k+1})$ and $\Theta(\frac{1}{n}(\log n)^{2k+1})$ for bounded and subexponential priors, respectively, attainable by estimators mimicking those that achieve optimal regret for the mean estimation problem (Robbins, mininum distance, ERM). Our analysis identifies the approximation-theoretic origin of this improvement: smooth functions can be well-approximated by low-degree polynomials, whereas Lipschitz functions require dense polynomial approximations, incurring a $\frac{1}{k}$ loss for degree $k$ polynomial approximation. The results reveal a sharp hierarchy in the difficulty of empirical Bayes problems: ranging from slow, logarithmic deconvolution to near-parametric convergence for smooth posterior functionals, and establish new connections between nonparametric empirical Bayes theory, polynomial approximation, and statistical inverse problems. Finally, we complement our analysis with a lower bound of $\Omega(\frac 1n (\frac{\log n}{\log \log n})^{k+1})$ (bounded priors) and $\Omega(\frac 1n (\log n)^{k + 1})$ (subgaussian priors) for the normal means model.