Evidential Calibration of Confidence Intervals

TL;DR

This paper introduces a new method for calibrating confidence intervals using evidence-based support intervals derived from Bayes factors, enabling more intuitive and informative inferences.

Contribution

It presents a novel approach to calibrate confidence intervals through support intervals based on Bayes factors, including methods to handle prior specification and future study planning.

Findings

Support intervals can be interpreted as likelihood ratios under specified priors.

Minimum support intervals can be computed to approximate confidence intervals without explicit prior specification.

Application to clinical trial data demonstrates the method's practical utility.

Abstract

We present a novel and easy-to-use method for calibrating error-rate based confidence intervals to evidence-based support intervals. Support intervals are obtained from inverting Bayes factors based on a parameter estimate and its standard error. A $k$ support interval can be interpreted as "the observed data are at least $k$ times more likely under the included parameter values than under a specified alternative". Support intervals depend on the specification of prior distributions for the parameter under the alternative, and we present several types that allow different forms of external knowledge to be encoded. We also show how prior specification can to some extent be avoided by considering a class of prior distributions and then computing so-called minimum support intervals which, for a given class of priors, have a one-to-one mapping with confidence intervals. We also illustrate how the sample size of a future study can be determined based on the concept of support. Finally, we show how the bound for the type I error rate of Bayes factors leads to a bound for the coverage of support intervals. An application to data from a clinical trial illustrates how support intervals can lead to inferences that are both intuitive and informative.