Beyond the Two-Trials Rule

TL;DR

This paper explores alternative statistical methods for drug approval beyond the traditional two-trials rule, focusing on controlling error rates when considering up to three studies, and recommends Edgington's method for its balance of power and simplicity.

Contribution

It introduces and evaluates less-known p-value combination methods for up to three studies, extending the two-trials rule to improve power while maintaining error control.

Findings

Edgington's method offers a good balance of power and error control.

Compared methods differ in partial Type-I error rate and efficiency.

Sequential assessment strategies can be effectively implemented.

Abstract

The two-trials rule for drug approval requires "at least two adequate and well-controlled studies, each convincing on its own, to establish effectiveness". This is usually implemented by requiring two significant pivotal trials and is the standard regulatory requirement to provide evidence for a new drug's efficacy. However, there is need to develop suitable alternatives to this rule for a number of reasons, including the possible availability of data from more than two trials. I consider the case of up to 3 studies and stress the importance to control the partial Type-I error rate, where only some studies have a true null effect, while maintaining the overall Type-I error rate of the two-trials rule, where all studies have a null effect. Some less-known $p$-value combination methods are useful to achieve this: Pearson's method, Edgington's method and the recently proposed harmonic mean $\chi^2$-test. I study their properties and discuss how they can be extended to a sequential assessment of success while still ensuring overall Type-I error control. I compare the different methods in terms of partial Type-I error rate, project power and the expected number of studies required. Edgington's method is eventually recommended as it is easy to implement and communicate, has only moderate partial Type-I error rate inflation but substantially increased project power.