Frequentist tests for Bayesian models

TL;DR

This paper introduces frequentist analogues of chi-square and F tests tailored for Bayesian models, enabling assessment of model fit and experiment consistency, with practical examples demonstrating their effectiveness in detecting inconsistencies and measuring tension.

Contribution

It proposes novel frequentist tests for Bayesian models, providing tools for goodness-of-fit and consistency evaluation not previously available.

Findings

Tests detect inconsistency between experiments with shared parameters.

Quantitative measure of tension between different experiments.

Effective in identifying model-data conflicts.

Abstract

Analogues of the frequentist chi-square and F tests are proposed for testing goodness-of-fit and consistency for Bayesian models. Simple examples exhibit these tests' detection of inconsistency between consecutive experiments with identical parameters, when the first experiment provides the prior for the second. In a related analysis, a quantitative measure is derived for judging the degree of tension between two different experiments with partially overlapping parameter vectors.