Targeted Parameter Estimation for Robust Empirical Bayes Ranking

TL;DR

This paper introduces a new empirical Bayes ranking method that optimizes the estimation of cluster-level outcomes, improving robustness and accuracy over traditional approaches in applications like healthcare and education.

Contribution

It proposes a novel ranking procedure based on estimating covariate-adjusted percentiles and introduces an unbiased estimator for expected ranking loss, enhancing robustness and performance.

Findings

Outperforms traditional methods in simulation studies.

Provides more accurate and robust rankings.

Demonstrates effectiveness with real-world school test score data.

Abstract

Ordering the expected outcomes across a collection of clusters after performing a covariate adjustment commonly arises in many applied settings, such as healthcare provider evaluation. Regression parameters in such covariate adjustment models are frequently estimated by maximum likelihood or through other criteria that do not directly evaluate the quality of the rankings resulting from using a particular set of parameter estimates. In this article, we propose both a novel empirical Bayes ranking procedure and an associated estimation approach for finding the regression parameters of the covariate adjustment model. By building our ranking approach around estimating approximate percentiles of the covariate-adjusted cluster-level means, we are able to develop manageable expressions for the expected ranking squared-error loss associated with any choice of the covariate-adjustment model parameters, and we harness this to generate a novel unbiased estimator for this expected loss. Minimization of this unbiased estimator directly leads to a novel ranking procedure that is often more robust than conventional empirical Bayes ranking methods. Through a series of simulation studies, we show that our approach consistently delivers improved ranking squared-error performance relative to competing methods, such as posterior expected ranks and ranking the components of the best linear unbiased predictor. Estimating rankings using our method is illustrated with an example from a longitudinal study evaluating test scores across a large group of schools.