Discriminative Mutual Information Estimators for Channel Capacity Learning

TL;DR

This paper introduces CORTICAL, a novel framework that uses discriminative mutual information estimation and cooperative learning to automatically determine the capacity of memoryless communication channels, overcoming the challenge of closed-form solutions.

Contribution

The paper presents a new methodology (DIME) for estimating mutual information directly from discriminators and integrates it into a cooperative framework (CORTICAL) for channel capacity learning.

Findings

High accuracy in capacity estimation demonstrated in simulations.

The framework effectively learns optimal input distributions for channels.

Discriminator-based mutual information estimation is viable for this task.

Abstract

Channel capacity plays a crucial role in the development of modern communication systems as it represents the maximum rate at which information can be reliably transmitted over a communication channel. Nevertheless, for the majority of channels, finding a closed-form capacity expression remains an open challenge. This is because it requires to carry out two formidable tasks a) the computation of the mutual information between the channel input and output, and b) its maximization with respect to the signal distribution at the channel input. In this paper, we address both tasks. Inspired by implicit generative models, we propose a novel cooperative framework to automatically learn the channel capacity, for any type of memory-less channel. In particular, we firstly develop a new methodology to estimate the mutual information directly from a discriminator typically deployed to train adversarial networks, referred to as discriminative mutual information estimator (DIME). Secondly, we include the discriminator in a cooperative channel capacity learning framework, referred to as CORTICAL, where a discriminator learns to distinguish between dependent and independent channel input-output samples while a generator learns to produce the optimal channel input distribution for which the discriminator exhibits the best performance. Lastly, we prove that a particular choice of the cooperative value function solves the channel capacity estimation problem. Simulation results demonstrate that the proposed method offers high accuracy.