A note on conditional densities, Bayes' rule, and recent criticisms of Bayesian inference

TL;DR

This paper clarifies the proper use of conditional densities in Bayesian inference, addresses recent criticisms claiming inconsistencies, and demonstrates that these criticisms are based on errors and misunderstandings.

Contribution

It provides an accessible explanation of conditional densities, a rigorous measure-theoretic roadmap, and a critique of recent criticisms of Bayesian inference.

Findings

Conditional densities are well-defined when properly understood.

Recent criticisms contain mathematical errors and deviate from Bayesian principles.

The authors defend the consistency of Bayesian inference against these criticisms.

Abstract

When performing Bayesian inference, we frequently need to work with conditional probability densities. For example, the posterior function is the conditional density of the parameters given the data. Some might worry that conditional densities are ill-defined, considering that for a continuous random variable $Y$, the event $\{Y=y\}$ has probability zero, meaning the formula $\mathbb{P}(A|B)=\mathbb{P}(A\cap B)/\mathbb{P}(B)$ is inapplicable. In reality, when we work with conditional densities, we never condition directly on the zero-probability event $\{Y=y\}$; rather, we first condition on the random variable $Y$, and then we may plug in an observed value $y$. The first purpose of our article is to provide an exposition on conditional densities that elaborates on this point. While we have aimed to make this explanation accessible, we follow it with a roadmap of the measure theory needed to make it rigorous. A recent preprint (arXiv:2411.13570) has expressed the concern that probability densities are ill-defined and that as a result Bayes' theorem cannot be used, and they provide examples that allegedly demonstrate inconsistencies in the Bayesian framework. The second purpose of our article is to investigate their claims. We contend that the examples given in their work do not demonstrate any inconsistencies; we find that there are mathematical errors and that they deviate significantly from the Bayesian framework.