Is Repeated Bayesian Interim Analysis Consequence-Free?

TL;DR

This paper investigates the effects of repeated Bayesian interim analyses in clinical trials, revealing that some properties remain unaffected while others are significantly influenced by prior choices and sequential testing.

Contribution

It provides a theoretical and numerical assessment of Bayesian interim monitoring impacts, clarifying when properties are invariant or affected, and emphasizes prior specification importance.

Findings

Matched priors preserve bias and coverage.

FDR, ATIE, and MSE are affected even with matched priors.

Differing priors significantly impact all operating characteristics.

Abstract

Interim analyses are vital in clinical trials for early decision-making. While frequentist implications are well-established, the consequences of repeated Bayesian interim monitoring for efficacy, specifically regarding multiplicity, remain contentious. This article provides theoretical justification and numerical evidence evaluating the impact of such designs on bias, mean squared error (MSE), credible interval coverage, false discovery rate (FDR), and average Type I error (ATIE). Our findings show that when the inferential prior matches the data-generating prior, sequential efficacy stopping does not bias the posterior mean or degrade credible interval coverage. However, even under this ``matched" condition, the FDR, ATIE, and MSE are significantly altered. In the more practically relevant scenario where the inferential and data-generating priors differ, all aforementioned operating characteristics, including estimation bias and coverage, are substantially impacted. These results reconcile long-standing conflicting arguments regarding Bayesian multiplicity. We demonstrate that while some Bayesian properties are invariant to sequential looks, others are not. Our work underscores the necessity of thoughtful prior specification and comprehensive evaluation of frequentist-Bayesian operating characteristics to ensure reliable inference in adaptive trial designs.