Robust Bayesian inference via coarsening

TL;DR

This paper proposes a robust Bayesian inference method that conditions on a neighborhood of the empirical distribution, using likelihood tempering to improve robustness against model misspecification, with practical implementations and theoretical insights.

Contribution

It introduces a coarsening approach to Bayesian inference that enhances robustness by conditioning on neighborhoods, enabling simple likelihood tempering and analytical solutions with conjugate priors.

Findings

Method improves robustness to model violations

Likelihood tempering simplifies implementation

Applicable to mixture, autoregressive, and regression models

Abstract

The standard approach to Bayesian inference is based on the assumption that the distribution of the data belongs to the chosen model class. However, even a small violation of this assumption can have a large impact on the outcome of a Bayesian procedure. We introduce a simple, coherent approach to Bayesian inference that improves robustness to perturbations from the model: rather than condition on the data exactly, one conditions on a neighborhood of the empirical distribution. When using neighborhoods based on relative entropy estimates, the resulting "coarsened" posterior can be approximated by simply tempering the likelihood---that is, by raising it to a fractional power---thus, inference is often easily implemented with standard methods, and one can even obtain analytical solutions when using conjugate priors. Some theoretical properties are derived, and we illustrate the approach with real and simulated data, using mixture models, autoregressive models of unknown order, and variable selection in linear regression.