A Practical Analysis Procedure on Generalizing Comparative Effectiveness in the Randomized Clinical Trial to the Real-world Trialeligible Population

TL;DR

This paper presents a practical procedure to extend RCT findings to the broader real-world population using advanced statistical methods, addressing external validity issues and robustness against unmeasured confounders.

Contribution

It introduces a generalized estimation procedure combining AIPSW and bias assessment for better external validity in clinical trial analysis.

Findings

SXC is less effective than Losartan in early weeks

No significant difference at week 8 in the trial-eligible population

Results are robust against unmeasured confounders

Abstract

When evaluating the effectiveness of a drug, a Randomized Controlled Trial (RCT) is often considered the gold standard due to its perfect randomization. While RCT assures strong internal validity, its restricted external validity poses challenges in extending treatment effects to the broader real-world population due to possible heterogeneity in covariates. In this paper, we introduce a procedure to generalize the RCT findings to the real-world trial-eligible population based on the adaption of existing statistical methods. We utilized the augmented inversed probability of sampling weighting (AIPSW) estimator for the estimation and omitted variable bias framework to assess the robustness of the estimate against the assumption violation caused by potentially unmeasured confounders. We analyzed an RCT comparing the effectiveness of lowering hypertension between Songling Xuemaikang Capsule (SXC), a traditional Chinese medicine (TCM), and Losartan as an illustration. The generalization results indicated that although SXC is less effective in lowering blood pressure than Losartan on week 2, week 4, and week 6, there is no statistically significant difference among the trial-eligible population at week 8, and the generalization is robust against potential unmeasured confounders.