Generalizing trial evidence to target populations in non-nested designs: Applications to AIDS clinical trials

TL;DR

This paper develops and applies statistical methods to generalize causal effects from AIDS clinical trials to broader populations, addressing challenges of non-random sampling and trial design.

Contribution

It introduces strategies for point and variance estimation in non-nested trial designs, demonstrating that estimating propensity scores does not inflate variance, with practical applications and simulations.

Findings

Estimators perform well in finite samples

Estimating propensity scores does not increase variance

Methods successfully applied to ACTG trial data

Abstract

Comparative effectiveness evidence from randomized trials may not be directly generalizable to a target population of substantive interest when, as in most cases, trial participants are not randomly sampled from the target population. Motivated by the need to generalize evidence from two trials conducted in the AIDS Clinical Trials Group (ACTG), we consider weighting, regression and doubly robust estimators to estimate the causal effects of HIV interventions in a specified population of people living with HIV in the USA. We focus on a non-nested trial design and discuss strategies for both point and variance estimation of the target population average treatment effect. Specifically in the generalizability context, we demonstrate both analytically and empirically that estimating the known propensity score in trials does not increase the variance for each of the weighting, regression and doubly robust estimators. We apply these methods to generalize the average treatment effects from two ACTG trials to specified target populations and operationalize key practical considerations. Finally, we report on a simulation study that investigates the finite-sample operating characteristics of the generalizability estimators and their sandwich variance estimators.