Capacity of Random Channels with Large Alphabets

TL;DR

This paper analyzes the capacity of large alphabet discrete memoryless channels with random transition matrices, showing convergence to a specific entropy-based limit and an exponential rate under certain conditions, with applications in experiment design.

Contribution

It establishes the asymptotic capacity of large random channels and demonstrates exponential convergence, extending understanding of channel behavior with random matrices.

Findings

Capacity converges to Ent(V)/E[V] as n→∞

Convergence is almost sure and in L^2

Capacity approaches the limit exponentially fast under certain conditions

Abstract

We consider discrete memoryless channels with input alphabet size $n$ and output alphabet size $m$, where $m=$ceil$(\gamma n)$ for some constant $\gamma>0$. The channel transition matrix consists of entries that, before being normalised, are independent and identically distributed nonnegative random variables $V$ and such that $E[(V \log V)^2]<\infty$. We prove that in the limit as $n\to \infty$ the capacity of such a channel converges to $Ent(V) / E[V]$ almost surely and in $L^2$, where $Ent(V):= E[V\log V]-E[V] \log E[V]$ denotes the entropy of $V$. We further show that, under slightly different model assumptions, the capacity of these random channels converges to this asymptotic value exponentially in $n$. Finally, we present an application in the context of Bayesian optimal experiment design.