Generalized Pinsker Inequality for Bregman Divergences of Negative Tsallis Entropies

TL;DR

This paper establishes a sharp, generalized Pinsker inequality for Bregman divergences derived from negative Tsallis entropies, linking divergence to total variation with explicit constants, aiding probabilistic prediction and online learning.

Contribution

It derives a novel, sharp lower bound for Bregman divergences of negative Tsallis entropies in terms of total variation, with explicit constants for all parameters.

Findings

Proves a generalized Pinsker inequality for Tsallis-based divergences.

Determines explicit optimal constants for the inequality.

Extends Pinsker inequality to a broader class of divergences.

Abstract

The Pinsker inequality lower bounds the Kullback--Leibler divergence $D_{\textrm{KL}}$ in terms of total variation and provides a canonical way to convert $D_{\textrm{KL}}$ control into $\lVert \cdot \rVert_1$-control. Motivated by applications to probabilistic prediction with Tsallis losses and online learning, we establish a generalized Pinsker inequality for the Bregman divergences $D_\alpha$ generated by the negative $\alpha$-Tsallis entropies -- also known as $\beta$-divergences. Specifically, for any $p$, $q$ in the relative interior of the probability simplex $\Delta^K$, we prove the sharp bound \[ D_\alpha(p\Vert q) \ge \frac{C_{\alpha,K}}{2}\cdot \|p-q\|_1^2, \] and we determine the optimal constant $C_{\alpha,K}$ explicitly for every choice of $(\alpha,K)$.