Alternating Optimization Approach for Computing $\alpha$-Mutual Information and $\alpha$-Capacity

TL;DR

This paper introduces alternating optimization algorithms for efficiently computing $oldsymbol{ extit{ extalpha}}$-mutual information and capacity, leveraging variational characterizations, with the Sibson-based method showing the fastest convergence.

Contribution

It develops novel AO algorithms for $ extit{ extalpha}$-MI and capacity based on various variational characterizations, including Sibson, Arimoto, Augustin--Csisz{\'a}r, and Lapidoth--Pfister.

Findings

Sibson MI-based AO algorithm converges fastest.

Comparison of different variational characterizations.

Algorithms effectively compute $ extit{ extalpha}$-capacity.

Abstract

This study presents alternating optimization (AO) algorithms for computing $\alpha$-mutual information ($\alpha$-MI) and $\alpha$-capacity based on variational characterizations of $\alpha$-MI using a reverse channel. Specifically, we derive several variational characterizations of Sibson, Arimoto, Augustin--Csisz{\' a}r, and Lapidoth--Pfister MI and introduce novel AO algorithms for computing $\alpha$-MI and $\alpha$-capacity; their performances for computing $\alpha$-capacity are also compared. The comparison results show that the AO algorithm based on the Sibson MI's characterization has the fastest convergence speed.