The analytic dually flat space of the mixture family of two prescribed distinct Cauchy distributions

TL;DR

This paper explores the geometric structure of mixture families, specifically deriving a closed-form expression for the Jensen-Shannon divergence between mixtures of two distinct Cauchy distributions, advancing understanding of their information geometry.

Contribution

It provides the first closed-form formula for the differential entropy and Jensen-Shannon divergence for mixtures of two prescribed Cauchy distributions.

Findings

Closed-form differential entropy for the mixture family of two Cauchy distributions.

Explicit Jensen-Shannon divergence formula for these mixtures.

Enhanced understanding of the geometric structure of this mixture family.

Abstract

A smooth and strictly convex function on an open convex domain induces both (1) a Hessian manifold with respect to the standard flat Euclidean connection, and (2) a dually flat space of information geometry. We first review these constructions and illustrate how to instantiate them for (a) full regular exponential families from their partition functions, (b) regular homogeneous cones from their characteristic functions, and (c) mixture families from their Shannon negentropy functions. Although these structures can be explicitly built for many common examples of the first two classes, the differential entropy of a continuous statistical mixture with distinct prescribed density components sharing the same support is hitherto not known in closed form, hence forcing implementations of mixture family manifolds in practice using Monte Carlo sampling. In this work, we report a notable exception: The family of mixtures defined as the convex combination of two prescribed and distinct Cauchy distributions. As a byproduct, we report closed-form formula for the Jensen-Shannon divergence between two mixtures of two prescribed Cauchy components.