A probabilistic view on predictive constructions for Bayesian learning

TL;DR

This paper explores a predictive approach to Bayesian learning that bypasses priors by directly selecting predictive strategies, providing a unifying framework, clarifying misunderstandings, and introducing new strategies like generalized Pólya urns and change points.

Contribution

It offers a comprehensive review of the predictive approach to Bayesian learning, clarifies its theoretical foundations, and introduces novel strategies for predictive modeling.

Findings

Unifies various predictive strategies under a common framework

Introduces new strategies involving Pólya urns and change points

Determines the distribution of data sequences for these strategies

Abstract

Given a sequence $X=(X_1,X_2,\ldots)$ of random observations, a Bayesian forecaster aims to predict $X_{n+1}$ based on $(X_1,\ldots,X_n)$ for each $n\ge 0$. To this end, in principle, she only needs to select a collection $\sigma=(\sigma_0,\sigma_1,\ldots)$, called ``strategy" in what follows, where $\sigma_0(\cdot)=P(X_1\in\cdot)$ is the marginal distribution of $X_1$ and $\sigma_n(\cdot)=P(X_{n+1}\in\cdot\mid X_1,\ldots,X_n)$ the $n$-th predictive distribution. Because of the Ionescu-Tulcea theorem, $\sigma$ can be assigned directly, without passing through the usual prior/posterior scheme. One main advantage is that no prior probability is to be selected. In a nutshell, this is the predictive approach to Bayesian learning. A concise review of the latter is provided in this paper. We try to put such an approach in the right framework, to make clear a few misunderstandings, and to provide a unifying view. Some recent results are discussed as well. In addition, some new strategies are introduced and the corresponding distribution of the data sequence $X$ is determined. The strategies concern generalized P\'olya urns, random change points, covariates and stationary sequences.