Tight Lower Bounds for $\alpha$-Divergences Under Moment Constraints and Relations Between Different $\alpha$

TL;DR

This paper establishes tight lower bounds for $oldsymbol{ extalpha}$-divergences under moment constraints and explores relations between different $oldsymbol{ extalpha}$-divergences, generalizing known relations like those between KL divergence and $oldsymbol{ extchi}^2$-divergence.

Contribution

It derives differential and integral relations among $oldsymbol{ extalpha}$-divergences and identifies conditions under which binary divergences attain these bounds.

Findings

Derived relations between $oldsymbol{ extalpha}$-divergences.

Established tight lower bounds under mean and variance constraints.

Identified conditions for divergences to attain bounds, including KL, Hellinger, and $oldsymbol{ extchi}^2$.

Abstract

The $\alpha$-divergences include the well-known Kullback-Leibler divergence, Hellinger distance and $\chi^2$-divergence. In this paper, we derive differential and integral relations between the $\alpha$-divergences that are generalizations of the relation between the Kullback-Leibler divergence and the $\chi^2$-divergence. We also show tight lower bounds for the $\alpha$-divergences under given means and variances. In particular, we show a necessary and sufficient condition such that the binary divergences, which are divergences between probability measures on the same $2$-point set, always attain lower bounds. Kullback-Leibler divergence, Hellinger distance, and $\chi^2$-divergence satisfy this condition.