Analysis for the Slow Convergence in Arimoto Algorithm

TL;DR

This paper analyzes the convergence speed of the Arimoto algorithm, identifying conditions for different convergence orders and providing new insights into its $1/N$ convergence rate based on derivatives of the Kullback-Leibler divergence.

Contribution

It introduces a novel analysis of the $1/N$ convergence order of the Arimoto algorithm using Taylor expansion and derivatives of the Kullback-Leibler divergence.

Findings

Convergence speed depends on derivatives of the Kullback-Leibler divergence.

Conditions for exponential and $1/N$ convergence are clarified.

Theoretical convergence rates are validated with channel matrix examples.

Abstract

In this paper, we investigate the convergence speed of the Arimoto algorithm. By analyzing the Taylor expansion of the defining function of the Arimoto algorithm, we will clarify the conditions for the exponential or $1/N$ order convergence and calculate the convergence speed. We show that the convergence speed of the $1/N$ order is evaluated by the derivatives of the Kullback-Leibler divergence with respect to the input probabilities. The analysis for the convergence of the $1/N$ order is new in this paper. Based on the analysis, we will compare the convergence speed of the Arimoto algorithm with the theoretical values obtained in our theorems for several channel matrices.