Jeffrey's update rule as a minimizer of Kullback-Leibler divergence

TL;DR

This paper provides a concise proof that Jeffrey's update rule minimizes the Kullback-Leibler divergence between observations and predictions in Bayesian updating, enhancing theoretical understanding of its optimality.

Contribution

It offers a simplified, high-level proof of Jeffrey's rule as a divergence minimizer, improving clarity over previous derivations.

Findings

Jeffrey's update reduces Kullback-Leibler divergence.

The proof is more concise and high-level than previous versions.

Supports Jeffrey's rule as an optimal Bayesian update method.

Abstract

In this paper, we show a more concise and high level proof than the original one, derived by researcher Bart Jacobs, for the following theorem: in the context of Bayesian update rules for learning or updating internal states that produce predictions, the relative entropy between the observations and the predictions is reduced when applying Jeffrey's update rule to update the internal state.