Posterior risk of modular and semi-modular Bayesian inference

TL;DR

This paper analyzes the bias-variance trade-off in modular Bayesian inference, introduces a semi-modular posterior that optimally balances this trade-off, and proves its superiority over the cut posterior in terms of posterior risk.

Contribution

It provides the first formal formulation of the bias-variance trade-off in cutting feedback and proposes a new semi-modular posterior that improves inference accuracy.

Findings

Semi-modular posterior outperforms the cut posterior in posterior risk.

Point inferences under the cut posterior are inadmissible.

The new method is demonstrated through multiple examples.

Abstract

Modular Bayesian methods perform inference in models that are specified through a collection of coupled sub-models, known as modules. These modules often arise from modelling different data sources or from combining domain knowledge from different disciplines. ``Cutting feedback'' is a Bayesian inference method that ensures misspecification of one module does not affect inferences for parameters in other modules, and produces what is known as the cut posterior. However, choosing between the cut posterior and the standard Bayesian posterior is challenging. When misspecification is not severe, cutting feedback can greatly increase posterior uncertainty without a large reduction of estimation bias, leading to a bias-variance trade-off. This trade-off motivates semi-modular posteriors, which interpolate between standard and cut posteriors based on a tuning parameter. In this work, we provide the first precise formulation of the bias-variance trade-off that is present in cutting feedback, and we propose a new semi-modular posterior that takes advantage of it. Under general regularity conditions, we prove that this semi-modular posterior is more accurate than the cut posterior according to a notion of posterior risk. An important implication of this result is that point inferences made under the cut posterior are inadmissable. The new method is demonstrated in a number of examples.