Strong Asymptotic Composition Theorems for Mutual Information Measures

TL;DR

This paper analyzes the asymptotic behavior of various mutual information measures, showing they grow exponentially to a limit with a universal exponent, providing insights into information flow in noisy systems.

Contribution

It provides strong asymptotic composition theorems for Sibson, Arimoto mutual informations, and maximal leakage, characterizing their exponential growth rates for large observation sets.

Findings

Mutual information measures grow exponentially with the size of observations.

The exponential growth rate (exponent) is independent of the specific measure and order.

Measures converge to a limit that depends on the measure and order.

Abstract

We characterize the growth of the Sibson and Arimoto mutual informations and $\alpha$-maximal leakage, of any order that is at least unity, between a random variable and a growing set of noisy, conditionally independent and identically-distributed observations of the random variable. Each of these measures increases exponentially fast to a limit that is order- and measure-dependent, with an exponent that is order- and measure-independent.