Parameter Estimation of Mutual Information Maximized Channels

TL;DR

This paper introduces two algorithms for jointly estimating the parameters and capacity-achieving input distribution of a mutual information maximized channel using output observations, outperforming naive methods.

Contribution

The paper proposes bilevel fixed-point and augmented Lagrangian algorithms based on Blahut--Arimoto conditions for efficient joint estimation of channel parameters and input distributions.

Findings

Both algorithms successfully recover true channel parameters and input distributions.

Naive maximum-likelihood approach fails to recover the true parameters under mutual information constraints.

Empirical results validate the effectiveness of the proposed methods.

Abstract

We study the problem of estimating a parametric discrete memoryless channel \( p(y \mid x; \boldsymbol{\theta}) \) when the transmitter selects its input distribution \( \pi \) to maximize mutual information under the true parameter \( \boldsymbol{\theta}^* \). Using only i.i.d.\ observations of the channel output, we aim to jointly estimate the capacity-achieving input distribution \( \boldsymbol{\pi}^* \) and the true channel parameter \( \boldsymbol{\theta}^* \). In general, recovery of \( \boldsymbol{\pi}^* \) and \( \boldsymbol{\theta}^* \) can be challenging. To that end, we propose two efficient algorithms based on the Blahut--Arimoto (BA) optimality conditions: (i) a bilevel fixed-point method and (ii) an augmented Lagrangian method. Empirical results demonstrate that both proposed algorithms successfully recover the true \( \boldsymbol{\theta}^* \) and \( \boldsymbol{\pi}^* \), whereas a naive maximum-likelihood approach that ignores the mutual-information maximization constraint fails to do so.