Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effects

TL;DR

This paper introduces RATE metrics, a flexible and statistically rigorous way to evaluate how well treatment prioritization rules identify individuals who benefit most, applicable across various study designs.

Contribution

It proposes a family of RATE metrics for assessing treatment prioritization rules, providing theoretical guarantees and unifying existing metrics like the Qini coefficient.

Findings

RATE metrics are agnostic to rule derivation method

Central limit theorem enables asymptotic inference

Application to aspirin targeting for stroke patients

Abstract

There are a number of available methods for selecting whom to prioritize for treatment, including ones based on treatment effect estimation, risk scoring, and hand-crafted rules. We propose rank-weighted average treatment effect (RATE) metrics as a simple and general family of metrics for comparing and testing the quality of treatment prioritization rules. RATE metrics are agnostic as to how the prioritization rules were derived, and only assess how well they identify individuals that benefit the most from treatment. We define a family of RATE estimators and prove a central limit theorem that enables asymptotically exact inference in a wide variety of randomized and observational study settings. RATE metrics subsume a number of existing metrics, including the Qini coefficient, and our analysis directly yields inference methods for these metrics. We showcase RATE in the context of a number of applications, including optimal targeting of aspirin to stroke patients.