Leveraging Population Outcomes to Improve the Generalization of Experimental Results

TL;DR

This paper introduces a novel residualized weighting method that leverages observational population data to improve the efficiency and robustness of causal effect estimation in randomized experiments, especially when the sample differs from the target population.

Contribution

The paper proposes a post-residualized weighting approach that uses outcome predictions from observational data to enhance population effect estimates without requiring correct model specification.

Findings

Efficiency gains demonstrated through simulations.

Method remains consistent under standard assumptions.

Application to job training experiments shows practical benefits.

Abstract

Generalizing causal estimates in randomized experiments to a broader target population is essential for guiding decisions by policymakers and practitioners in the social and biomedical sciences. While recent papers developed various weighting estimators for the population average treatment effect (PATE), many of these methods result in large variance because the experimental sample often differs substantially from the target population, and estimated sampling weights are extreme. To improve efficiency in practice, we propose post-residualized weighting in which we use the outcome measured in the observational population data to build a flexible predictive model (e.g., machine learning methods) and residualize the outcome in the experimental data before using conventional weighting methods. We show that the proposed PATE estimator is consistent under the same assumptions required for existing weighting methods, importantly without assuming the correct specification of the predictive model. We demonstrate the efficiency gains from this approach through simulations and our application based on a set of job training experiments.