A simple probabilistic construction yielding generalized entropies and divergences, escort distributions and q-Gaussians

TL;DR

This paper introduces a simple probabilistic framework for generalized entropies, divergences, escort distributions, and q-Gaussians, revealing new insights into their interrelations and dynamics.

Contribution

It provides a novel probabilistic description of escort distributions and their paths, connecting them with Rényi divergence, Fisher information, and deriving generalized q-Gaussian solutions.

Findings

Escort-path dynamics relate to Jeffreys' divergence.

Rényi divergence naturally emerges in the framework.

Optimal distributions are generalized q-Gaussians.

Abstract

We give a simple probabilistic description of a transition between two states which leads to a generalized escort distribution. When the parameter of the distribution varies, it defines a parametric curve that we call an escort-path. The R\'enyi divergence appears as a natural by-product of the setting. We study the dynamics of the Fisher information on this path, and show in particular that the thermodynamic divergence is proportional to Jeffreys' divergence. Next, we consider the problem of inferring a distribution on the escort-path, subject to generalized moments constraints. We show that our setting naturally induces a rationale for the minimization of the R\'enyi information divergence. Then, we derive the optimum distribution as a generalized q-Gaussian distribution.