Constructive Disintegration and Conditional Modes

TL;DR

This paper develops mathematical tools for constructing disintegrations of measures, explores their properties on manifolds, and examines differences in modes of disintegrations versus restricted densities, with implications for Bayesian inference.

Contribution

It provides a comprehensive framework for constructing measure disintegrations, analyzes their modes, and highlights practical differences in Bayesian applications.

Findings

Disintegrations can be constructed using new mathematical tools.

Modes of disintegrations differ from modes of restricted densities.

Discrepancies have practical implications in Bayesian inference.

Abstract

Conditioning, the central operation in Bayesian statistics, is formalised by the notion of disintegration of measures. However, due to the implicit nature of their definition, constructing disintegrations is often difficult. A folklore result in machine learning conflates the construction of a disintegration with the restriction of probability density functions onto the subset of events that are consistent with a given observation. We provide a comprehensive set of mathematical tools which can be used to construct disintegrations and apply these to find densities of disintegrations on differentiable manifolds. Using our results, we provide a disturbingly simple example in which the restricted density and the disintegration density drastically disagree. Motivated by applications in approximate Bayesian inference and Bayesian inverse problems, we further study the modes of disintegrations. We show that the recently introduced notion of a "conditional mode" does not coincide in general with the modes of the conditional measure obtained through disintegration, but rather the modes of the restricted measure. We also discuss the implications of the discrepancy between the two measures in practice, advocating for the utility of both approaches depending on the modelling context.