An Estimator-Robust Design for Augmenting Randomized Controlled Trial with External Real-World Data

TL;DR

This paper introduces a robust method for integrating external real-world data into randomized controlled trials using adaptive targeted maximum likelihood estimation, enhancing treatment effect estimation accuracy.

Contribution

It proposes a matching-based sampling strategy to improve the robustness of A-TMLE when augmenting RCTs with external data, addressing data inconsistency issues.

Findings

Improved confidence interval coverage in simulations.

Shorter confidence intervals with the proposed sampling strategy.

Successful case study augmentation of a cardiovascular trial.

Abstract

Augmenting randomized controlled trials (RCTs) with external real-world data (RWD) has the potential to improve the finite sample efficiency of treatment effect estimators. We describe using adaptive targeted maximum likelihood estimation (A-TMLE) for estimating the average treatment effect (ATE) by decomposing the ATE estimand into two components: a pooled-ATE estimand that combines data from both the RCT and external sources, and a bias estimand that captures the conditional effect of RCT enrollment on the outcome. This approach views the RCT data as the reference and corrects for inconsistencies of any kind between the RCT and the external data source. Given the growing abundance of external RWD from modern electronic health records, determining the optimal strategy to select candidate external patients for data integration remains an open yet critical problem. In this work, we begin by analyzing the robustness property of the A-TMLE estimator and then propose a matching-based sampling strategy that improves the robustness of the estimator with respect to the target estimand. Our proposed strategy is outcome-blind and involves matching based on two one-dimensional scores: the trial enrollment score and the propensity score in the external data. We demonstrate in simulations that our sampling strategy improves the coverage and shortens the widths of confidence intervals produced by A-TMLE. We illustrate our method with a case study of augmenting the DEVOTE cardiovascular safety trial by using the Optum Clinformatics claims database.