Common Randomness Generation from Sources with Countable Alphabet

TL;DR

This paper derives a single-letter formula for the capacity of common randomness generation between two parties observing correlated sources on countably infinite alphabets, addressing the challenges posed by infinite entropy properties.

Contribution

It provides the first rigorous single-letter capacity formula for common randomness generation with countably infinite sources, extending finite alphabet results.

Findings

Derived a single-letter formula for CR capacity on countably infinite alphabets

Proved the formula rigorously despite entropy discontinuity issues

Addressed the challenge of extending finite alphabet properties to infinite cases

Abstract

We study a standard two-source model for common randomness (CR) generation in which Alice and Bob generate a common random variable with high probability of agreement by observing independent and identically distributed (i.i.d.) samples of correlated sources on countably infinite alphabets. The two parties are additionally allowed to communicate as little as possible over a noisy memoryless channel. In our work, we give a single-letter formula for the CR capacity for the proposed model and provide a rigorous proof of it. This is a challenging scenario because some of the finite alphabet properties, namely of the entropy can not be extended to the countably infinite case. Notably, it is known that the Shannon entropy is in fact discontinuous at all probability distributions with countably infinite support.