On the Search Algorithm for the Output Distribution that Achieves the Channel Capacity

TL;DR

This paper introduces an iterative projection algorithm to find the output distribution that achieves the channel capacity of a discrete memoryless channel, leveraging geometric structures and Amari's α-geometry.

Contribution

It proposes a novel algorithm based on affine projections and geometric insights to compute the capacity-achieving output distribution.

Findings

Algorithm successfully finds the capacity-achieving distribution.

Geometric approach links channel capacity to smallest enclosing circle problem.

Method extends Euclidean geometric concepts to information geometry.

Abstract

We consider a search algorithm for the output distribution that achieves the channel capacity of a discrete memoryless channel. We will propose an algorithm by iterated projections of an output distribution onto affine subspaces in the set of output distributions. The problem of channel capacity has a similar geometric structure as that of smallest enclosing circle for a finite number of points in the Euclidean space. The metric in the Euclidean space is the Euclidean distance and the metric in the space of output distributions is the Kullback-Leibler divergence. We consider these two problems based on Amari's $\alpha$-geometry. Then, we first consider the smallest enclosing circle in the Euclidean space and develop an algorithm to find the center of the smallest enclosing circle. Based on the investigation, we will apply the obtained algorithm to the problem of channel capacity.