Beyond the classical type I error: Bayesian metrics for Bayesian designs using informative priors

TL;DR

This paper introduces Bayesian metrics beyond classical type I error for evaluating Bayesian clinical trial designs with informative priors, focusing on false positive risk assessment and regulatory considerations.

Contribution

It proposes probabilistic metrics like average type I error and pre-posterior probability to better evaluate Bayesian designs with informative priors, especially when borrowing data.

Findings

Average type I error can be asymptotically controlled when priors match.

Bayesian metrics provide a more nuanced risk assessment than classical error rates.

Application examples demonstrate practical use in regulatory discussions.

Abstract

There is growing interest in Bayesian clinical trial designs with informative prior distributions, e.g. for extrapolation of adult data to pediatrics, or use of external controls. While the classical type I error is commonly used to evaluate such designs, it cannot be strictly controlled and it is acknowledged that other metrics may be more appropriate. We focus on two common situations - borrowing control data or information on the treatment contrast - and discuss several fully probabilistic metrics to evaluate the risk of false positive conclusions. Each metric requires specification of a design prior, which can differ from the analysis prior and permits understanding of the behaviour of a Bayesian design under scenarios where the analysis prior differs from the true data generation process. The metrics include the average type I error and the pre-posterior probability of a false positive result. We show that, when borrowing control data, the average type I error is asymptotically (in certain cases strictly) controlled when the analysis and design prior coincide. We illustrate use of these Bayesian metrics with real applications, and discuss how they could facilitate discussions between sponsors, regulators and other stakeholders about the appropriateness of Bayesian borrowing designs for pivotal studies.