Extension of the Blahut-Arimoto algorithm for maximizing directed information

TL;DR

This paper extends the Blahut-Arimoto algorithm to maximize directed information, enabling capacity estimation of channels with delayed feedback using an iterative optimization approach.

Contribution

The paper introduces a novel extension of the Blahut-Arimoto algorithm for directed information maximization, incorporating backward index time maximization and providing convergence guarantees.

Findings

Algorithm converges to the optimal directed information value.

Provides bounds that approach the channel capacity with feedback.

Demonstrates effectiveness through numerical examples.

Abstract

We extend the Blahut-Arimoto algorithm for maximizing Massey's directed information. The algorithm can be used for estimating the capacity of channels with delayed feedback, where the feedback is a deterministic function of the output. In order to do so, we apply the ideas from the regular Blahut-Arimoto algorithm, i.e., the alternating maximization procedure, onto our new problem. We provide both upper and lower bound sequences that converge to the optimum value. Our main insight in this paper is that in order to find the maximum of the directed information over causal conditioning probability mass function (PMF), one can use a backward index time maximization combined with the alternating maximization procedure. We give a detailed description of the algorithm, its complexity, the memory needed, and several numerical examples.