Large-Scale Estimation under Unknown Heteroskedasticity

TL;DR

This paper develops nonparametric empirical Bayes methods for estimating heterogeneous means and variances, providing nearly optimal estimators with applications to evaluating teacher quality.

Contribution

It extends Tweedie's formulae to unknown heteroskedastic settings and proposes feasible estimators with near-parametric regret bounds.

Findings

Allowing for heteroskedasticity improves teacher quality estimates.

Feasible estimators achieve near-optimal regret bounds.

Application demonstrates importance of variance heterogeneity in practice.

Abstract

This paper studies nonparametric empirical Bayes methods in a heterogeneous parameters framework that features unknown means and variances. We provide extended Tweedie's formulae that express the (infeasible) optimal estimators of heterogeneous parameters, such as unit-specific means or quantiles, in terms of the density of certain sufficient statistics. These are used to propose feasible versions with nearly parametric regret bounds of the order of $(\log n)^\kappa / n$. The estimators are employed in a study of teachers' value-added, where we find that allowing for heterogeneous variances across teachers is crucial for delivery optimal estimates of teacher quality and detecting low-performing teachers.