Data-Driven Estimation of Capacity Upper Bounds

TL;DR

This paper introduces a data-driven method to estimate upper bounds on the capacity of memoryless channels with unknown laws, using neural estimators and dual representations, validated through numerical experiments.

Contribution

A novel algorithm leveraging dual capacity representation and neural estimators to estimate channel capacity upper bounds from data.

Findings

Estimated bounds closely match true capacity or known lower bounds.

Method effectively handles unknown channel laws and continuous output alphabets.

Numerical results demonstrate the approach's accuracy across different channels.

Abstract

We consider the problem of estimating an upper bound on the capacity of a memoryless channel with unknown channel law and continuous output alphabet. A novel data-driven algorithm is proposed that exploits the dual representation of capacity where the maximization over the input distribution is replaced with a minimization over a reference distribution on the channel output. To efficiently compute the required divergence maximization between the conditional channel and the reference distribution, we use a modified mutual information neural estimator that takes the channel input as an additional parameter. We numerically evaluate our approach on different memoryless channels and show empirically that the estimated upper bounds closely converge either to the channel capacity or to best-known lower bounds.