On the computability of conditional probability
Nathanael L. Ackerman, Cameron E. Freer, Daniel M. Roy

TL;DR
This paper investigates the limits of computability in probabilistic inference, showing that some conditional probabilities are noncomputable, but identifying conditions under which they are computable, such as with certain noise models.
Contribution
It demonstrates that not all conditional probabilities are computable, even with computable joint distributions, and provides conditions ensuring computability in practical scenarios.
Findings
Existence of noncomputable conditional distributions from computable joint distributions.
Conditional distributions are computable when measurements include independent computable noise.
Constructed examples encode the halting problem within conditional probabilities.
Abstract
As inductive inference and machine learning methods in computer science see continued success, researchers are aiming to describe ever more complex probabilistic models and inference algorithms. It is natural to ask whether there is a universal computational procedure for probabilistic inference. We investigate the computability of conditional probability, a fundamental notion in probability theory and a cornerstone of Bayesian statistics. We show that there are computable joint distributions with noncomputable conditional distributions, ruling out the prospect of general inference algorithms, even inefficient ones. Specifically, we construct a pair of computable random variables in the unit interval such that the conditional distribution of the first variable given the second encodes the halting problem. Nevertheless, probabilistic inference is possible in many common modeling…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
