On variance estimation for generalizing from a trial to a target population

TL;DR

This paper develops and compares variance estimation methods for generalizing RCT results to broader populations using inverse probability weighting, focusing on robustness to heterogeneity in treatment effects.

Contribution

It introduces a model-free variance estimator for inverse probability weighted treatment effect estimation and compares its performance with model-based methods.

Findings

The proposed variance estimator performs well in simulations.

Model-free methods show robustness to heterogeneous treatment effects.

Comparison highlights advantages over traditional model-based approaches.

Abstract

Randomized controlled trials (RCTs) provide strong internal validity compared with observational studies. However, selection bias threatens the external validity of randomized trials. Thus, RCT results may not apply to either broad public policy populations or narrow populations, such as specific insurance pools. Some researchers use propensity scores (PSs) to generalize results from an RCT to a target population. In this scenario, a PS is defined as the probability of participating in the trial conditioning on observed covariates. We study a model-free inverse probability weighted estimator (IPWE) of the average treatment effect in a target population with data from a randomized trial. We present variance estimators and compare the performance of our method with that of model-based approaches. We examine the robustness of the model-free estimators to heterogeneous treatment effects.