A scaling law from discrete to continuous solutions of channel capacity problems in the low-noise limit

TL;DR

This paper derives a new scaling law showing how the optimal number of symbols in an analog communication channel increases with capacity as noise decreases, revealing a rational exponent in the low-noise limit.

Contribution

It introduces a novel rational scaling law for the growth of optimal symbols in channel capacity problems as noise approaches zero.

Findings

Number of symbols scales as log K ~ I^{4/3} in the low-noise limit.

Scaling law applies to various problems maximizing capacity over continuous distributions.

Provides a theoretical link between discrete and continuous optimal coding strategies.

Abstract

An analog communication channel typically achieves its full capacity when the distribution of inputs is discrete, composed of just K symbols, such as voltage levels or wavelengths. As the effective noise level goes to zero, for example by sending the same message multiple times, it is known that optimal codes become continuous. Here we derive a scaling law for the optimal number of symbols in this limit, finding a novel rational scaling exponent. The number of symbols in the optimal code grows as $\log K \sim I^{4/3}$, where the channel capacity I increases with decreasing noise. The same scaling applies to other problems equivalent to maximizing channel capacity over a continuous distribution.