Sibson $\alpha$-Mutual Information and Its Variational Representations

TL;DR

This paper surveys and extends Sibson α-mutual information, introducing variational representations that lead to new inequalities and broadening its applications in statistical learning, hypothesis testing, and estimation.

Contribution

It introduces variational representations for Sibson α-mutual information and derives new inequalities, expanding its theoretical and practical applications.

Findings

Derived generalized Transportation-Cost inequalities

Established Fano-type inequalities using Sibson α-mutual information

Extended applications to learning theory and Bayesian risk

Abstract

Information measures can be constructed from R\'enyi divergences much like mutual information from Kullback-Leibler divergence. One such information measure is known as Sibson $\alpha$-mutual information and has received renewed attention recently in several contexts: concentration of measure under dependence, statistical learning, hypothesis testing, and estimation theory. In this paper, we survey and extend the state of the art. In particular, we introduce variational representations for Sibson $\alpha$-mutual information and employ them in each described context to derive novel results. Namely, we produce generalized Transportation-Cost inequalities and Fano-type inequalities. We also present an overview of known applications, spanning from learning theory and Bayesian risk to universal prediction.