Stochastic Interpretation for the Arimoto Algorithm

TL;DR

This paper offers a stochastic interpretation of the Arimoto algorithm, linking its iterative process to maximizing Gallager exponents for codebooks, paving the way for online channel input adaptation.

Contribution

It introduces a novel stochastic perspective on the Arimoto algorithm, connecting its iterations to codeword types that optimize Gallager exponents.

Findings

The next distribution in the Arimoto algorithm matches the type of codewords maximizing the Gallager exponent.

This interpretation provides a foundation for developing online channel input adaptation mechanisms.

The approach links iterative optimization to stochastic codebook selection processes.

Abstract

The Arimoto algorithm computes the Gallager function $\max_Q {E}_{0}^{}(\rho,Q)$ for a given channel ${P}_{}^{}(y \,|\, x)$ and parameter $\rho$, by means of alternating maximization. Along the way, it generates a sequence of input distributions ${Q}_{1}^{}(x)$, ${Q}_{2}^{}(x)$, ... , that converges to the maximizing input ${Q}_{}^{*}(x)$. We propose a stochastic interpretation for the Arimoto algorithm. We show that for a random (i.i.d.) codebook with a distribution ${Q}_{k}^{}(x)$, the next distribution ${Q}_{k+1}^{}(x)$ in the Arimoto algorithm is equal to the type (${Q}'$) of the feasible transmitted codeword that maximizes the conditional Gallager exponent (conditioned on a specific transmitted codeword type ${Q}'$). This interpretation is a first step toward finding a stochastic mechanism for on-line channel input adaptation.