Exponential Family Models from Bayes' Theorem under Expectation Constraints

TL;DR

This paper demonstrates that Bayesian updating with expectation constraints naturally yields exponential family distributions, providing an alternative to entropy-based inference methods without using variational principles.

Contribution

It introduces a Bayesian updating approach based on expectation constraints that produces exponential family posteriors without relying on entropy or variational methods.

Findings

Bayesian updating under expectation constraints results in exponential family posteriors.

The method is conceptually distinct from entropy-based inference.

It offers a complete alternative to entropic inference methods.

Abstract

It is shown that a consistent application of Bayesian updating from a prior probability density to a posterior using evidence in the form of expectation constraints leads to exactly the same results as the application of the maximum entropy principle, namely a posterior belonging to the exponential family. The Bayesian updating procedure presented in this work is not expressed as a variational principle, and does not involve the concept of entropy. Therefore it conceptually constitutes a complete alternative to entropic methods of inference.