Getting Wiser from Multiple Data: Probabilistic Updating according to Jeffrey and Pearl

TL;DR

This paper compares Jeffrey and Pearl's probabilistic updating methods for multiple evidence pieces, illustrating their differences and properties, and relates them to variational free energy updates in cognitive predictive processing.

Contribution

It provides an overview and comparison of Jeffrey and Pearl's updating methods for multiple data, highlighting their advantages over variational free energy updates.

Findings

Jeffrey and Pearl's updates outperform variational free energy in reducing divergence.

Both updating methods incorporate multiple evidence pieces effectively.

VFE updating may not always decrease errors as expected.

Abstract

In probabilistic updating one transforms a prior distribution in the light of given evidence into a posterior distribution, via what is called conditioning, updating, belief revision or inference. This is the essence of learning, as Bayesian updating. It will be illustrated via a physical model involving (adapted) water flows through pipes with different diameters. Bayesian updating makes us wiser, in the sense that the posterior distribution makes the evidence more likely than the prior, since it incorporates the evidence. Things are less clear when one wishes to learn from multiple pieces of evidence / data. It turns out that there are (at least) two forms of updating for this, associated with Jeffrey and Pearl. The difference is not always clearly recognised. This paper provides an introduction and an overview in the setting of discrete probability theory. It starts from an elementary question, involving multiple pieces of evidence, that has been sent to a small group academic specialists. Their answers show considerable differences. This is used as motivation and starting point to introduce the two forms of updating, of Jeffrey and Pearl, for multiple inputs and to elaborate their properties. In the end the account is related to so-called variational free energy (VFE) update in the cognitive theory of predictive processing. It is shown that both Jeffrey and Pearl outperform VFE updating and that VFE updating need not decrease divergence - that is correct errors - as it is supposed to do.