Updating Probabilities with Data and Moments

TL;DR

This paper develops a framework using Maximum Entropy methods to update probabilities with data and moment constraints, addressing the order of processing non-commuting information and illustrating with die toss examples.

Contribution

It derives a canonical form for the posterior distribution when updating with both data and moments, and discusses the handling of non-commuting constraints.

Findings

Derived a general posterior form for combined data and moment updates.

Analyzed the impact of constraint ordering on the updating process.

Provided detailed examples with die tosses illustrating the method.

Abstract

We use the method of Maximum (relative) Entropy to process information in the form of observed data and moment constraints. The generic "canonical" form of the posterior distribution for the problem of simultaneous updating with data and moments is obtained. We discuss the general problem of non-commuting constraints, when they should be processed sequentially and when simultaneously. As an illustration, the multinomial example of die tosses is solved in detail for two superficially similar but actually very different problems.