An exposition of the false confidence theorem

TL;DR

This paper discusses the false confidence theorem, highlighting its implications for Bayesian inference and demonstrating through examples how certain models can lead to misleading confidence statements.

Contribution

It provides a detailed exposition of the false confidence theorem and explores conditions under which Bayesian procedures may produce problematic inferences.

Findings

Models with non-linear parameter functions exhibit more extreme false confidence.

Examples show the paradoxical nature of false confidence in Bayesian inference.

Understanding the theorem helps identify when Bayesian methods may be misleading.

Abstract

A recent paper presents the "false confidence theorem" (FCT) which has potentially broad implications for statistical inference using Bayesian posterior uncertainty. This theorem says that with arbitrarily large (sampling/frequentist) probability, there exists a set which does \textit{not} contain the true parameter value, but which has arbitrarily large posterior probability. Since the use of Bayesian methods has become increasingly popular in applications of science, engineering, and business, it is critically important to understand when Bayesian procedures lead to problematic statistical inferences or interpretations. In this paper, we consider a number of examples demonstrating the paradoxical nature of false confidence to begin to understand the contexts in which the FCT does (and does not) play a meaningful role in statistical inference. Our examples illustrate that models involving marginalization to non-linear, not one-to-one functions of multiple parameters play a key role in more extreme manifestations of false confidence.