Geometrical interpretation and improvements of the Blahut-Arimoto's algorithm

TL;DR

This paper offers a geometric perspective on the Blahut-Arimoto algorithm, interpreting it as a projection-based method and proposing an improved version with faster convergence.

Contribution

It introduces a geometric interpretation of the Blahut-Arimoto algorithm and develops a proximal point algorithm variant with enhanced convergence properties.

Findings

The geometric interpretation clarifies the algorithm's structure.

The proximal point version converges faster than the original.

Improved convergence compared to other variants.

Abstract

The paper first recalls the Blahut Arimoto algorithm for computing the capacity of arbitrary discrete memoryless channels, as an example of an iterative algorithm working with probability density estimates. Then, a geometrical interpretation of this algorithm based on projections onto linear and exponential families of probabilities is provided. Finally, this understanding allows also to propose to write the Blahut-Arimoto algorithm, as a true proximal point algorithm. it is shown that the corresponding version has an improved convergence rate, compared to the initial algorithm, as well as in comparison with other improved versions.