Bayesian forecasting with information theory

TL;DR

This paper introduces a Bayesian forecasting approach based on information theory, specifically mutual information, to evaluate future experiments' ability to discriminate theories and discover new laws without relying on fiducial parameters.

Contribution

It presents a novel Bayesian method using mutual information applicable to any probability distribution, enhancing parameter estimation and model selection for future experiments.

Findings

Mutual information effectively assesses experiment power.

The method is independent of true theory assumptions.

Propositions link Bayesian practices with information theory.

Abstract

Forecasting techniques for assessing the power of future experiments to discriminate between theories or discover new laws of nature are of great interest in many areas of science. In this paper, we introduce a Bayesian forecasting method using information theory. We argue that mutual information is a suitable quantity to study in this context. Besides being Bayesian, this proposal has the advantage of not relying on the choice of fiducial parameters, describing the "true" theory (which is a priori unknown), and is applicable to any probability distribution. We demonstrate that the proposed method can be used for parameter estimation and model selection, both of which are of interest concerning future experiments. We argue that mutual information has plausible interpretation in both situations. In addition, we state a number of propositions that offer information-theoretic meaning to some of the Bayesian practices such as performing multiple experiments, combining different datasets, and marginalization.