A Framework for Generalization and Transportation of Causal Estimates Under Covariate Shift

TL;DR

This paper introduces a framework and estimators for generalizing and transporting causal effect estimates from experimental samples to target populations under covariate shift, addressing external validity concerns.

Contribution

It decomposes extrapolation bias into components and proposes re-weighting estimators for better generalization and transportation of causal effects, implemented in an open-source R package.

Findings

Estimators effectively reduce bias under covariate shift.

Simulation studies demonstrate improved accuracy of causal effect estimates.

Diagnostics help assess the performance of the proposed methods.

Abstract

Randomized experiments are an excellent tool for estimating internally valid causal effects with the sample at hand, but their external validity is frequently debated. While classical results on the estimation of Population Average Treatment Effects (PATE) implicitly assume random selection into experiments, this is typically far from true in many medical, social-scientific, and industry experiments. When the experimental sample is different from the target sample along observable or unobservable dimensions, experimental estimates may be of limited use for policy decisions. We begin by decomposing the extrapolation bias from estimating the Target Average Treatment Effect (TATE) using the Sample Average Treatment Effect (SATE) into covariate shift, overlap, and effect modification components, which researchers can reason about in order to diagnose the severity of extrapolation bias. Next, We cast covariate shift as a sample selection problem and propose estimators that re-weight the doubly-robust scores from experimental subjects to estimate treatment effects in the overall sample (=: generalization) or in an alternate target sample (=: transportation). We implement these estimators in the open-source R package causalTransportR and illustrate its performance in a simulation study and discuss diagnostics to evaluate its performance.