Martingale posterior distributions

TL;DR

This paper introduces the martingale posterior distribution, a new Bayesian uncertainty quantification method that directly estimates the distribution of statistics without relying on traditional likelihood or prior, using predictive resampling.

Contribution

It proposes a novel martingale posterior framework that bypasses the need for likelihood and prior, enabling direct Bayesian inference on statistics via predictive resampling.

Findings

Provides a new predictive methodology for density estimation, regression, and classification.

Demonstrates the use of martingale properties to derive Bayesian uncertainty without traditional priors.

Introduces computational schemes for implementing the martingale posterior.

Abstract

The prior distribution on parameters of a sampling distribution is the usual starting point for Bayesian uncertainty quantification. In this paper, we present a different perspective which focuses on missing observations as the source of statistical uncertainty, with the parameter of interest being known precisely given the entire population. We argue that the foundation of Bayesian inference is to assign a distribution on missing observations conditional on what has been observed. In the conditionally i.i.d. setting with an observed sample of size $n$, the Bayesian would thus assign a predictive distribution on the missing $Y_{n+1:\infty}$ conditional on $Y_{1:n}$, which then induces a distribution on the parameter. Demonstrating an application of martingales, Doob shows that choosing the Bayesian predictive distribution returns the conventional posterior as the distribution of the parameter. Taking this as our cue, we relax the predictive machine, avoiding the need for the predictive to be derived solely from the usual prior to posterior to predictive density formula. We introduce the \textit{martingale posterior distribution}, which returns Bayesian uncertainty directly on any statistic of interest without the need for the likelihood and prior, and this distribution can be sampled through a computational scheme we name \textit{predictive resampling}. To that end, we introduce new predictive methodologies for multivariate density estimation, regression and classification that build upon recent work on bivariate copulas.