Benchmarking covariate-adjustment strategies for randomized clinical trials

TL;DR

This study empirically benchmarks 18 covariate adjustment strategies across 50 RCTs, showing that simple, transparent methods often outperform complex machine-learning approaches in improving statistical efficiency.

Contribution

It provides the first large-scale empirical comparison of covariate adjustment methods, guiding best practices for routine RCT analysis.

Findings

Covariate adjustment reduces variance by median 13.3% for continuous outcomes.

Machine-learning methods with default settings do not outperform simple models.

Parsimonious regression approaches are stable and effective across sample sizes.

Abstract

Covariate adjustment is widely recommended to improve statistical efficiency in randomized clinical trials (RCTs), yet empirical evidence comparing available strategies remains limited. This lack of real-world evaluation leaves unresolved practical questions about which adjustment methods to use and which covariates to include. To address this gap, we conduct a large-scale empirical benchmarking using individual-level data from 50 publicly accessible RCTs comprising 29,094 participants and 574 treatment-outcome pairs. We evaluate 18 analytical strategies formed by combining six estimators-including classical regression, inverse probability weighting, and machine-learning methods-with three covariate-selection rules. Across diverse therapeutic areas, covariate adjustment consistently improves precision, yielding median variance reductions of 13.3% relative to unadjusted analyses for continuous outcomes and 4.6% for binary outcomes. However, machine-learning algorithms implemented with default hyperparameter settings do not yield efficiency gains beyond simple linear models. Parsimonious regression approaches, such as analysis of covariance, deliver stable, reproducible performance even in moderate sample sizes. Together, these findings provide the first large-scale empirical evidence that transparent and parsimonious covariate adjustment is sufficient and often preferable for routine RCT analysis. All curated datasets and analysis code are openly released as a reproducible benchmark resource to support future clinical research and methodological development.