[Invited Discussion] Randomization Tests to Address Disruptions in Clinical Trials: A Report from the NISS Ingram Olkin Forum Series on Unplanned Clinical Trial Disruptions

TL;DR

This paper discusses the use of randomization tests to address disruptions in clinical trials caused by external events, comparing their robustness to other estimation methods like Targeted Learning, and emphasizes the need for theoretical validation of their effectiveness.

Contribution

It advocates for the application of randomization tests in disrupted clinical trials and critically examines their theoretical advantages over targeted learning approaches.

Findings

Randomization tests are robust to violations caused by trial disruptions.

Targeted Learning methods are valid but may not be optimal in disrupted settings.

The paper calls for theoretical demonstration of the superiority of randomization tests.

Abstract

Disruptions in clinical trials may be due to external events like pandemics, warfare, and natural disasters. Resulting complications may lead to unforeseen intercurrent events (events that occur after treatment initiation and affect the interpretation of the clinical question of interest or the existence of the measurements associated with it). In Uschner et al. (2023), several example clinical trial disruptions are described: treatment effect drift, population shift, change of care, change of data collection, and change of availability of study medication. A complex randomized controlled trial (RCT) setting with (planned or unplanned) intercurrent events is then described, and randomization tests are presented as a means for non-parametric inference that is robust to violations of assumption typically made in clinical trials. While estimation methods like Targeted Learning (TL) are valid in such settings, we do not see where the authors make the case that one should be going for a randomization test in such disrupted RCTs. In this discussion, we comment on the appropriateness of TL and the accompanying TL Roadmap in the context of disrupted clinical trials. We highlight a few key articles related to the broad applicability of TL for RCTs and real-world data (RWD) analyses with intercurrent events. We begin by introducing TL and motivating its utility in Section 2, and then in Section 3 we provide a brief overview of the TL Roadmap. In Section 4 we recite the example clinical trial disruptions presented in Uschner et al. (2023), discussing considerations and solutions based on the principles of TL. We request in an authors' rejoinder a clear theoretical demonstration with specific examples in this setting that a randomization test is the only valid inferential method relative to one based on following the TL Roadmap.