Things Bayes can't do

TL;DR

This paper demonstrates fundamental limitations of Bayesian predictors in forecasting conditional probabilities, showing that for some predictor sets, non-Bayesian methods outperform Bayesian ones in asymptotic regret.

Contribution

It establishes that there exist predictor sets where optimal non-Bayesian predictors outperform Bayesian predictors, highlighting a fundamental limitation of Bayesian methods in certain probabilistic forecasting scenarios.

Findings

Bayesian predictors can have linear regret on some predictor sets.

Non-Bayesian predictors can achieve sublinear regret in the same settings.

Contrasts with previous results where Bayesian predictors matched optimal performance.

Abstract

The problem of forecasting conditional probabilities of the next event given the past is considered in a general probabilistic setting. Given an arbitrary (large, uncountable) set C of predictors, we would like to construct a single predictor that performs asymptotically as well as the best predictor in C, on any data. Here we show that there are sets C for which such predictors exist, but none of them is a Bayesian predictor with a prior concentrated on C. In other words, there is a predictor with sublinear regret, but every Bayesian predictor must have a linear regret. This negative finding is in sharp contrast with previous results that establish the opposite for the case when one of the predictors in $C$ achieves asymptotically vanishing error. In such a case, if there is a predictor that achieves asymptotically vanishing error for any measure in C, then there is a Bayesian predictor that also has this property, and whose prior is concentrated on (a countable subset of) C.