A Training-Based Mutual Information Lower Bound for Large-Scale Systems

TL;DR

This paper introduces a new mutual information lower bound for large-scale systems that leverages derivatives of conditional entropy, enabling analysis of training effects without explicit parameter estimation.

Contribution

It presents a novel mutual information lower bound applicable to large-scale systems that does not require explicit estimation of unknown parameters.

Findings

The bound can be computed using derivatives of a conditional entropy function.

It provides a practical step-by-step process for calculating the bound.

Comparison with classical bounds demonstrates its effectiveness.

Abstract

We provide a mutual information lower bound that can be used to analyze the effect of training in models with unknown parameters. For large-scale systems, we show that this bound can be calculated using the difference between two derivatives of a conditional entropy function. The bound does not require explicit estimation of the unknown parameters. We provide a step-by-step process for computing the bound, and provide an example application. A comparison with known classical mutual information bounds is provided.