Generalised exponential families and associated entropy functions

TL;DR

This paper introduces a generalized framework for exponential families based on a variational principle, revealing how different entropy functions define distinct families and extending classical statistical results.

Contribution

It proposes a new generalized notion of exponential families using variational principles and explores their properties, including the role of escort probabilities.

Findings

Different entropy functions lead to distinct exponential families.

The Cramer-Rao inequality becomes an equality with escort probabilities.

The framework generalizes classical exponential family results.

Abstract

A generalised notion of exponential families is introduced. It is based on the variational principle, borrowed from statistical physics. It is shown that inequivalent generalised entropy functions lead to distinct generalised exponential families. The well-known result that the inequality of Cramer and Rao becomes an equality in the case of an exponential family can be generalised. However, this requires the introduction of escort probabilities.