ETZ: A Modeling Principle for Confirmability of Drug-Development Studies

TL;DR

This paper introduces ETZ, a modeling principle that enhances the confirmability of drug development studies by using confidence sets and the CBQ framework, improving decision-making from Phase 2 to Phase 3.

Contribution

It proposes a novel methodological shift replacing traditional hypothesis testing with confidence set inference and introduces the CBQ framework for more reliable transitioning decisions.

Findings

Confidence sets improve transition decisions in drug studies.

CBQ framework reduces incorrect rejection probabilities.

ETZ quantifies variability impacts to guide investment in variability reduction.

Abstract

Transitioning from Phase 2 to Phase 3 in drug development, at a rate of $\approx$40%, is the most stringent among phase transitions (Hay et al. (2014)). Yet, success rate at Phase 3 leading to approval is only $\approx$50% (Arrowsmith (2011b)). To improve Confirmability, we propose a methodological shift: replacing multiple hypothesis testing with inference based on confidence sets, and substituting conventional power and sample size calculations with a Confidently Bounded Quantile (CBQ) framework. Our confidence set inferences to answer the questions of whether to transition to a Confirmatory study as well as what to designate as the endpoint in that study. Construction of our directed confidence sets follows the Partitioning Principle, taking the best of each of Pivoting and Neyman Confidence Set Construction. Rooted in Tukey's Confidently Bounded Allowance (CBA) (Tukey (1994a)), our proposed CBQ makes the transitioning decision following the Correct and Useful Inference principle in Hsu (1996). CBQ removes from "power" the probability of rejecting for wrong reasons, eliminating the need for informal discounting in power calculation that has existed in the biopharmaceutical industry. ETZ, the modeling principle proposed in Wang et al. (2025), quantifies the impact of three variability components on confirmability. In repeated-measures RCTs, it separates within-subject and between-subject variability, further dividing the latter into baseline and trajectory components. This enables informed investment decisions for the sponsors on targeting variability reduction to improve confirmability. A Shiny-based Confirmability App supports all computations.