Closed-Form Power and Sample Size Calculations for Bayes Factors

TL;DR

This paper introduces closed-form and numerical methods for calculating power and sample size in Bayesian hypothesis testing using Bayes factors, avoiding computationally intensive simulations.

Contribution

It presents novel closed-form formulas and numerical approaches for Bayesian sample size determination under various priors, including practical tools like the R package bfpwr.

Findings

Closed-form formulas enable easy sample size calculation.

Numerical methods work under approximate normality assumptions.

Methods are demonstrated with case studies in medicine and psychology.

Abstract

Determining an appropriate sample size is a critical element of study design, and the method used to determine it should be consistent with the planned analysis. When the planned analysis involves Bayes factor hypothesis testing, the sample size is usually desired to ensure a sufficiently high probability of obtaining a Bayes factor indicating compelling evidence for a hypothesis, given that the hypothesis is true. In practice, Bayes factor sample size determination is typically performed using computationally intensive Monte Carlo simulation. Here, we summarize alternative approaches that enable sample size determination without simulation. We show how, under approximate normality assumptions, sample sizes can be determined numerically, and provide the R package bfpwr for this purpose. Additionally, we identify conditions under which sample sizes can even be determined in closed-form, resulting in novel, easy-to-use formulas that also help foster intuition, enable asymptotic analysis, and can also be used for hybrid Bayesian/likelihoodist design. Furthermore, we show how power and sample size can be computed without simulation for more complex analysis priors, such as Jeffreys-Zellner-Siow priors or non-local normal moment priors. Case studies from medicine and psychology illustrate how researchers can use our methods to design informative yet cost-efficient studies.