A new group of transformations related to the Kullback-Leibler and R\'enyi divergences and universal classes of monotone measures of statistical complexity

TL;DR

This paper introduces divergence transformations that interpolate between probability densities, showing they preserve monotonicity of Kullback-Leibler and Re9nyi divergences, with applications in density approximation and analysis.

Contribution

It presents a novel family of divergence transformations with a group structure, linking differential-escort transformations to measure deformation and divergence monotonicity.

Findings

Divergence transformations interpolate between densities while preserving divergence monotonicity.

The algebraic structure enables density deformation to maximize divergence.

Applications include density approximation and analysis of divergence behavior.

Abstract

In this work we introduce a family of transformations, named \textit{divergence transformations}, interpolating between any pair of probability density functions sharing the same support. We prove the remarkable property that the whole family of Kullback-Leibler and R\'enyi divergences evolves in a monotone way with respect to the transformation parameter. Moreover, fixing the reference density, we show that the divergence transformations enjoy a group structure and can be derived through the algebraic conjugation of the recently introduced differential-escort transformations and their relative counterparts. This algebraic structure allows us to deform any density function in such a way its divergence with respect a fixed reference density might also increase as much as possible. We also establish the monotonicity of composed measures involving the proper Kullback-Leibler and R\'enyi divergences as well as other recently introduced relative measures of moment and Fisher types. As applications, an approximation scheme of general density functions by simple functions is provided. In addition, we give a number of analytical and numerical examples of interest in both regimes of increasing and decreasing divergence.