Optimal multiple testing and design in clinical trials

TL;DR

This paper develops a framework for designing optimal multiple testing procedures in clinical trials, balancing power and error control, and introduces novel policies that outperform existing methods in simulations and real data.

Contribution

It formulates an explicit optimization approach for multiple hypothesis testing in clinical trials and derives new, more powerful procedures with desirable properties.

Findings

Derived optimal procedures for two hypotheses with monotonicity

Existing procedures like Hommel's are special cases of the framework

New policies demonstrate improved power in simulations and real data

Abstract

A central goal in designing clinical trials is to find the test that maximizes power (or equivalently minimizes required sample size) for finding a false null hypothesis subject to the constraint of type I error. When there is more than one test, such as in clinical trials with multiple endpoints, the issues of optimal design and optimal procedures become more complex. In this paper we address the question of how such optimal tests should be defined and how they can be found. We review different notions of power and how they relate to study goals, and also consider the requirements of type I error control and the nature of the procedures. This leads us to an explicit optimization problem with objective and constraints which describe its specific desiderata. We present a complete solution for deriving optimal procedures for two hypotheses, which have desired monotonicity properties, and are computationally simple. For some of the optimization formulations this yields optimal procedures that are identical to existing procedures, such as Hommel's procedure or the procedure of Bittman et al. (2009), while for others it yields completely novel and more powerful policies than existing ones. We demonstrate the nature of our novel policies and their improved power extensively in simulation and on the APEX study (Cohen et al., 2016).