Entropic Priors and Bayesian Model Selection

TL;DR

This paper shows how entropic priors derived from maximum relative entropy can improve Bayesian model selection by naturally penalizing overly sharp predictions, without needing to explicitly enumerate hypotheses.

Contribution

It introduces a formalism for entropic priors that simplifies model selection and addresses debates in cosmology without hypothesis enumeration.

Findings

Entropic priors weaken Bayesian Occam's Razor against sharp models.

Illustrated with a rigged-lottery example demonstrating the approach.

Potential application to cosmological debates on dark energy.

Abstract

We demonstrate that the principle of maximum relative entropy (ME), used judiciously, can ease the specification of priors in model selection problems. The resulting effect is that models that make sharp predictions are disfavoured, weakening the usual Bayesian "Occam's Razor". This is illustrated with a simple example involving what Jaynes called a "sure thing" hypothesis. Jaynes' resolution of the situation involved introducing a large number of alternative "sure thing" hypotheses that were possible before we observed the data. However, in more complex situations, it may not be possible to explicitly enumerate large numbers of alternatives. The entropic priors formalism produces the desired result without modifying the hypothesis space or requiring explicit enumeration of alternatives; all that is required is a good model for the prior predictive distribution for the data. This idea is illustrated with a simple rigged-lottery example, and we outline how this idea may help to resolve a recent debate amongst cosmologists: is dark energy a cosmological constant, or has it evolved with time in some way? And how shall we decide, when the data are in?