Divergence functions in dually flat spaces and their properties

TL;DR

This paper introduces new divergence functions in dually flat spaces, explores their geometric properties, and establishes inequalities that generalize classical results like Lin's inequality.

Contribution

It proposes two novel divergence functions in dually flat spaces, linking them to well-known divergences and extending geometric relations such as the law of cosines.

Findings

New divergence functions consistent with Jeffreys, Bhattacharyya, and Jensen-Shannon divergences.

Derived relational equations similar to Euclidean space laws.

Established an inequality generalizing Lin's inequality.

Abstract

In the field of statistics, many kind of divergence functions have been studied as an amount which measures the discrepancy between two probability distributions. In the differential geometrical approach in statistics (information geometry), dually flat spaces play a key role. In a dually flat space, there exist dual affine coordinate systems and strictly convex functions called potential and a canonical divergence is naturally introduced as a function of the affine coordinates and potentials. The canonical divergence satisfies a relational expression called triangular relation. This can be regarded as a generalization of the law of cosines in Euclidean space. In this paper, we newly introduce two kinds of divergences. The first divergence is a function of affine coordinates and it is consistent with the Jeffreys divergence for exponential or mixture families. For this divergence, we show that more relational equations and theorems similar to Euclidean space hold in addition to the law of cosines. The second divergences are functions of potentials and they are consistent with the Bhattacharyya distance for exponential families and are consistent with the Jensen-Shannon divergence for mixture families respectively. We derive an inequality between the the first and the second divergences and show that the inequality is a generalization of Lin's inequality.