# Predictively Consistent Prior Effective Sample Sizes

## TL;DR

This paper introduces a new method for calculating prior effective sample sizes that is predictively consistent, improving upon existing methods especially in non-conjugate Bayesian models, with applications in clinical trial design.

## Contribution

The paper proposes the expected local-information-ratio ESS, a novel measure that satisfies predictive consistency, correcting limitations of existing ESS methods in complex Bayesian settings.

## Key findings

- The new ESS method is predictively consistent across various models.
- Existing methods can produce inconsistent ESS estimates in non-conjugate models.
- Applications include prior ESS estimation from historical data and hierarchical subgroup analysis.

## Abstract

Determining the sample size of an experiment can be challenging, even more so when incorporating external information via a prior distribution. Such information is increasingly used to reduce the size of the control group in randomized clinical trials. Knowing the amount of prior information, expressed as an equivalent prior effective sample size (ESS), clearly facilitates trial designs. Various methods to obtain a prior's ESS have been proposed recently. They have been justified by the fact that they give the standard ESS for one-parameter exponential families. However, despite being based on similar information-based metrics, they may lead to surprisingly different ESS for non-conjugate settings, which complicates many designs with prior information. We show that current methods fail a basic predictive consistency criterion, which requires the expected posterior-predictive ESS for a sample of size $N$ to be the sum of the prior ESS and $N$. The expected local-information-ratio ESS is introduced and shown to be predictively consistent. It corrects the ESS of current methods, as shown for normally distributed data with a heavy-tailed Student-t prior and exponential data with a generalized Gamma prior. Finally, two applications are discussed: the prior ESS for the control group derived from historical data, and the posterior ESS for hierarchical subgroup analyses.