On Bayes' theorem for improper mixtures

TL;DR

This paper explores how improper priors can be rigorously interpreted within Bayesian inference by extending the parameter space to the power set, ensuring the validity of Bayes's theorem.

Contribution

It introduces a method to extend models with improper priors to the power set of the parameter space, satisfying Bayes's theorem under certain conditions.

Findings

Improper measures satisfying Kingman's condition can be viewed as probability measures on the power set.

Extension of models to the power set allows proper Bayesian updating with improper priors.

Under finiteness and non-interference conditions, improper Bayes procedures are justified.

Abstract

Although Bayes's theorem demands a prior that is a probability distribution on the parameter space, the calculus associated with Bayes's theorem sometimes generates sensible procedures from improper priors, Pitman's estimator being a good example. However, improper priors may also lead to Bayes procedures that are paradoxical or otherwise unsatisfactory, prompting some authors to insist that all priors be proper. This paper begins with the observation that an improper measure on Theta satisfying Kingman's countability condition is in fact a probability distribution on the power set. We show how to extend a model in such a way that the extended parameter space is the power set. Under an additional finiteness condition, which is needed for the existence of a sampling region, the conditions for Bayes's theorem are satisfied by the extension. Lack of interference ensures that the posterior distribution in the extended space is compatible with the original parameter space. Provided that the key finiteness condition is satisfied, this probabilistic analysis of the extended model may be interpreted as a vindication of improper Bayes procedures derived from the original model.