Estimating the Efficiency Gain of Covariate-Adjusted Analyses in Future Clinical Trials Using External Data

TL;DR

This paper introduces a framework to estimate how much covariate adjustment can improve efficiency in future clinical trials using external data, with methods applicable to various estimands and outcome observation scenarios.

Contribution

It develops a semiparametrically efficient estimator for relative efficiency and provides a practical approach to predict efficiency gains from covariate adjustment in future trials.

Findings

The estimator performs well in simulations.

Application to Covid-19 trials demonstrates practical utility.

Confidence intervals accurately reflect uncertainty.

Abstract

We present a general framework for using existing data to estimate the efficiency gain from using a covariate-adjusted estimator of a marginal treatment effect in a future randomized trial. We describe conditions under which it is possible to define a mapping from the distribution that generated the existing external data to the relative efficiency of a covariate-adjusted estimator compared to an unadjusted estimator. Under conditions, these relative efficiencies approximate the ratio of sample size needed to achieve a desired power. We consider two situations where the outcome is either fully or partially observed and several treatment effect estimands that are of particular interest in most trials. For each such estimand, we develop a semiparametrically efficient estimator of the relative efficiency that allows for the application of flexible statistical learning tools to estimate the nuisance functions and an analytic form of a corresponding Wald-type confidence interval. We also propose a double bootstrap scheme for constructing confidence intervals. We demonstrate the performance of the proposed methods through simulation studies and apply these methods to data to estimate the relative efficiency of using covariate adjustment in Covid-19 therapeutic trials.