Balancing Evidentiary Value and Sample Size of Adaptive Designs with Application to Animal Experiments

TL;DR

This paper introduces the experimental unit information index (EUII), a new measure combining power, error rates, and sample size to optimize evidentiary value in adaptive animal experiments.

Contribution

It proposes a novel EUII measure for quantifying evidentiary value, extending it to adaptive designs with early stopping, and demonstrates its application in animal research.

Findings

EUII depends only on effect size under the alternative asymptotically.

Application to group-sequential designs shows potential for sample size reduction.

Reanalysis of 2738 experiments illustrates possible savings in animal use.

Abstract

Reducing the number of experimental units is one of the three pillars of the 3R principles (Replace, Reduce, Refine) in animal research. At the same time, statistical error rates need to be controlled to enable reliable inferences and decisions. This paper proposes to adopt diagnostic likelihood ratios and the diagnostic odds ratio to statistical hypothesis tests and to adjust it for sample size to obtain a novel measure to quantify for the evidentiary value of one experimental unit. The experimental unit information index (EUII) is based on power, Type-I error and sample size, and has attractive interpretations both in terms of frequentist error rates and Bayesian posterior odds. We introduce the EUII in simple statistical test settings and show that its asymptotic value depends only on the assumed relative effect size under the alternative. We then extend the definition to adaptive designs where early stopping for efficacy or futility may cause reductions in sample size. Application to group-sequential designs show the usefulness of the approach when the goal is to maximize the evidentiary value of one experimental unit. A reanalysis of 2738 animal experiments with simulated results from (post-hoc) interim analyses illustrates the possible savings in sample size.