Externally Valid Policy Choice

TL;DR

This paper develops methods for estimating personalized treatment policies that remain effective across different populations, addressing the challenge of external validity and distributional shifts in outcomes and characteristics.

Contribution

It introduces robust policy estimation techniques that account for shifts in outcome distributions and highlights the role of treatment heterogeneity in external validity.

Findings

Welfare-maximizing policies are robust to certain outcome distribution shifts.

Methods for estimating policies resilient to joint distribution shifts.

Treatment heterogeneity influences external validity of policies.

Abstract

We consider the problem of estimating personalized treatment policies that are "externally valid" or "generalizable": they perform well in target populations that differ from the experimental (or training) population from which the data are sampled. We first show that welfare-maximizing policies for the experimental population are robust to a certain class of shifts in the distribution of potential outcomes between the experimental and target populations (holding characteristics fixed). We then develop methods for estimating policies that are robust to shifts in the joint distribution of outcomes and characteristics. In doing so, we highlight how treatment effect heterogeneity within the experimental population shapes external validity.