A Remark on the Assumptions of Bayes' Theorem

TL;DR

This paper clarifies the precise conditions under which Bayes' theorem for conditional densities holds, providing equivalent formulations and counterexamples to highlight the assumptions needed for its validity.

Contribution

It establishes equivalent conditions for the validity of Bayes' formula for conditional densities and discusses implications for adaptive estimation.

Findings

Equivalent conditions for the validity of Bayes' formula are identified.

Counterexamples demonstrate the nontriviality of assumptions.

Implications for sequential adaptive estimation are discussed.

Abstract

We formulate simple equivalent conditions for the validity of Bayes' formula for conditional densities. We show that for any random variables X and Y (with values in arbitrary measurable spaces), the following are equivalent: 1. X and Y have a joint density w.r.t. a product measure \mu x \nu, 2. P_{X,Y} << P_X x P_Y, (here P_{.} denotes the distribution of {.}) 3. X has a conditional density p(x | y) w.r.t. a sigma-finite measure \mu, 4. X has a conditional distribution P_{X|Y} such that P_{X|y} << P_X for all y, 5. X has a conditional distribution P_{X|Y} and a marginal density p(x) w.r.t. a measure \mu such that P_{X|y} << \mu for all y. Furthermore, given random variables X and Y with a conditional density p(y | x) w.r.t. \nu and a marginal density p(x) w.r.t. \mu, we show that Bayes' formula p(x | y) = p(y | x)p(x) / \int p(y | x)p(x)d\mu(x) yields a conditional density p(x | y) w.r.t. \mu if and only if X and Y satisfy the above conditions. Counterexamples illustrating the nontriviality of the results are given, and implications for sequential adaptive estimation are considered.