Upper Bounds on the Relative Entropy and R\'enyi Divergence as a Function of Total Variation Distance for Finite Alphabets

TL;DR

This paper derives improved upper bounds on relative entropy and Re9nyi divergence based on total variation distance for finite alphabet probability measures, enhancing previous bounds by Csisze1r and Talata.

Contribution

It introduces new, tighter upper bounds on relative entropy and Re9nyi divergence as functions of total variation distance for finite alphabet distributions.

Findings

Improved upper bound on relative entropy as a function of total variation distance.

Extended bounds to Re9nyi divergence of any non-negative order.

The bounds are tighter than previous results by Csisze1r and Talata.

Abstract

A new upper bound on the relative entropy is derived as a function of the total variation distance for probability measures defined on a common finite alphabet. The bound improves a previously reported bound by Csisz\'ar and Talata. It is further extended to an upper bound on the R\'enyi divergence of an arbitrary non-negative order (including $\infty$) as a function of the total variation distance.