Sensitivity analysis for an unobserved moderator in RCT-to-target-population generalization of treatment effects

TL;DR

This paper develops sensitivity analysis methods to assess how unobserved moderators affect the generalization of treatment effects from RCTs to target populations, especially when some moderators are unobserved or unmeasured.

Contribution

It introduces new sensitivity analysis techniques for unobserved moderators in RCT-to-population generalization, addressing both observed and unobserved moderators with multiple methods.

Findings

Three methods for unobserved moderators with observed outcome

Two methods for unobserved moderators without outcome data

Application to smoking cessation RCT with unobserved moderator analysis

Abstract

In the presence of treatment effect heterogeneity, the average treatment effect (ATE) in a randomized controlled trial (RCT) may differ from the average effect of the same treatment if applied to a target population of interest. If all treatment effect moderators are observed in the RCT and in a dataset representing the target population, we can obtain an estimate for the target population ATE by adjusting for the difference in the distribution of the moderators between the two samples. This paper considers sensitivity analyses for two situations: (1) where we cannot adjust for a specific moderator $V$ observed in the RCT because we do not observe it in the target population; and (2) where we are concerned that the treatment effect may be moderated by factors not observed even in the RCT, which we represent as a composite moderator $U$. In both situations, the outcome is not observed in the target population. For situation (1), we offer three sensitivity analysis methods based on (i) an outcome model, (ii) full weighting adjustment, and (iii) partial weighting combined with an outcome model. For situation (2), we offer two sensitivity analyses based on (iv) a bias formula and (v) partial weighting combined with a bias formula. We apply methods (i) and (iii) to an example where the interest is to generalize from a smoking cessation RCT conducted with participants of alcohol/illicit drug use treatment programs to the target population of people who seek treatment for alcohol/illicit drug use in the US who are also cigarette smokers. In this case a treatment effect moderator is observed in the RCT but not in the target population dataset.