A Perspective on Neural Capacity Estimation: Viability and Reliability

TL;DR

This paper evaluates the effectiveness of neural mutual information estimators across various communication channel scenarios, establishing a benchmark to assess their reliability and performance in capacity estimation tasks.

Contribution

It introduces a comprehensive benchmark for neural mutual information estimators across multiple channel models and compares the performance of MINE, SMILE, and DINE.

Findings

MINE provides the most reliable performance among tested estimators.

Benchmarking across diverse scenarios reveals strengths and limitations of NMIEs.

The study offers insights into the suitability of different estimators for capacity estimation.

Abstract

Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks. These estimators ar referred to as neural mutual information estimation (NMIE)s. NMIEs differ from other approaches as they are data-driven estimators. As such, they have the potential to perform well on a large class of capacity problems. In order to test the performance across various NMIEs, it is desirable to establish a benchmark encompassing the different challenges of capacity estimation. This is the objective of this paper. In particular, we consider three scenarios for benchmarking:i the classic AWGN channel, ii channels continuous inputs optical intensity and peak-power constrained AWGN channel iii channels with a discrete output, i.e., Poisson channel. We also consider the extension to the multi-terminal case with iv the AWGN and optical MAC models. We argue that benchmarking a certain NMIE across these four scenarios provides a substantive test of performance. In this paper we study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and directed information neural estimator (DINE). and provide insights on the performance of other methods as well. To summarize our benchmarking results, MINE provides the most reliable performance.