Validating Bayesian Inference Algorithms with Simulation-Based Calibration

TL;DR

This paper introduces simulation-based calibration (SBC), a comprehensive method for validating Bayesian inference algorithms by detecting inaccuracies and inconsistencies in complex models through graphical summaries.

Contribution

The paper presents SBC as a novel, general validation procedure for Bayesian algorithms that enhances the robustness of Bayesian workflows and software development.

Findings

SBC effectively detects inaccuracies in Bayesian computations.

Graphical summaries help diagnose specific issues in inference algorithms.

SBC is applicable to complex models requiring sophisticated implementations.

Abstract

Verifying the correctness of Bayesian computation is challenging. This is especially true for complex models that are common in practice, as these require sophisticated model implementations and algorithms. In this paper we introduce \emph{simulation-based calibration} (SBC), a general procedure for validating inferences from Bayesian algorithms capable of generating posterior samples. This procedure not only identifies inaccurate computation and inconsistencies in model implementations but also provides graphical summaries that can indicate the nature of the problems that arise. We argue that SBC is a critical part of a robust Bayesian workflow, as well as being a useful tool for those developing computational algorithms and statistical software.