Conditional Mutual Information Neural Estimator

TL;DR

This paper introduces a neural estimator for conditional mutual information in communication systems, providing a rigorous lower bound and addressing challenges related to conditional densities.

Contribution

It presents a novel neural-based estimator for conditional mutual information with a mathematically rigorous lower bound, improving on previous methods.

Findings

Provides a variational bound for the KL-divergence in the estimator.

Addresses challenges with conditional density functions in mutual information estimation.

Offers a mathematically rigorous lower bound for the estimator.

Abstract

Several recent works in communication systems have proposed to leverage the power of neural networks in the design of encoders and decoders. In this approach, these blocks can be tailored to maximize the transmission rate based on aggregated samples from the channel. Motivated by the fact that, in many communication schemes, the achievable transmission rate is determined by a conditional mutual information term, this paper focuses on neural-based estimators for this information-theoretic quantity. Our results are based on variational bounds for the KL-divergence and, in contrast to some previous works, we provide a mathematically rigorous lower bound. However, additional challenges with respect to the unconditional mutual information emerge due to the presence of a conditional density function which we address here.