Tight Bounds for Symmetric Divergence Measures and a New Inequality Relating $f$-Divergences

TL;DR

This paper establishes tight bounds for symmetric divergence measures based on total variation distance, introduces a new inequality for $f$-divergences, and applies these results to improve bounds in lossless source coding.

Contribution

It provides the first tight bounds for symmetric divergence measures in terms of total variation and introduces a novel inequality relating $f$-divergences.

Findings

Tight bounds for symmetric divergence measures are attained by simple probability distributions.

The new inequality for $f$-divergences enhances understanding of divergence relationships.

Application to lossless source coding refines existing bounds.

Abstract

Tight bounds for several symmetric divergence measures are introduced, given in terms of the total variation distance. Each of these bounds is attained by a pair of 2 or 3-element probability distributions. An application of these bounds for lossless source coding is provided, refining and improving a certain bound by Csisz\'ar. A new inequality relating $f$-divergences is derived, and its use is exemplified. The last section of this conference paper is not included in the recent journal paper that was published in the February 2015 issue of the IEEE Trans. on Information Theory (see arXiv:1403.7164), as well as some new paragraphs throughout the paper which are linked to new references.