On the proper treatment of improper distributions

TL;DR

This paper reviews a recent theoretical framework that incorporates improper distributions into probability theory, clarifying paradoxes and providing a solid foundation for their use in statistical applications.

Contribution

It introduces a rigorous theory including improper distributions, connecting to Renyi's conditional probability, and explains paradoxes in Bayesian statistics with practical examples.

Findings

Provides a theoretical basis for improper distributions

Explains Bayesian paradoxes using the new theory

Discusses convergence of proper to improper distributions

Abstract

The axiomatic foundation of probability theory presented by Kolmogorov has been the basis of modern theory for probability and statistics. In certain applications it is, however, necessary or convenient to allow improper (unbounded) distributions, which is often done without a theoretical foundation. The paper reviews a recent theory which includes improper distributions, and which is related to Renyi's theory of conditional probability spaces. It is in particular demonstrated how the theory leads to simple explanations of apparent paradoxes known from the Bayesian literature. Several examples from statistical practice with improper distributions are discussed in light of the given theoretical results, which also include a recent theory of convergence of proper distributions to improper ones.