Are adaptive allocation designs beneficial for improving power in binary response trials?

TL;DR

This paper investigates whether adaptive allocation designs improve the power of binary response trials, finding that balanced designs are often optimal or nearly so, questioning the necessity of adaptive methods.

Contribution

It provides a theoretical comparison of adaptive and fixed designs based on Pitman and Bahadur efficiency criteria, showing balanced designs are often optimal.

Findings

Balanced allocation is optimal under Pitman criterion.

Bahadur optimal allocation depends on unknown parameters.

Balanced design often matches or exceeds adaptive designs in power.

Abstract

We consider the classical problem of selecting the best of two treatments in clinical trials with binary response. The target is to find the design that maximizes the power of the relevant test. Many papers use a normal approximation to the power function and claim that Neyman allocation that assigns subjects to treatment groups according to the ratio of the responses' standard deviations, should be used. As the standard deviations are unknown, an adaptive design is often recommended. The asymptotic justification of this approach is arguable, since it uses the normal approximation in tails where the error in the approximation is larger than the estimated quantity. We consider two different approaches for optimality of designs that are related to Pitman and Bahadur definitions of relative efficiency of tests. We prove that the optimal allocation according to the Pitman criterion is the balanced allocation and that the optimal allocation according to the Bahadur approach depends on the unknown parameters. Exact calculations reveal that the optimal allocation according to Bahadur is often close to the balanced design, and the powers of both are comparable to the Neyman allocation for small sample sizes and are generally better for large experiments. Our findings have important implications to the design of experiments, as the balanced design is proved to be optimal or close to optimal and the need for the complications involved in following an adaptive design for the purpose of increasing the power of tests is therefore questionable.