A Generalized Expression for the Gradient of Mutual Information with the Application in Multiple Access Channels

TL;DR

This paper derives a comprehensive formula for the gradient of mutual information in stochastic systems, including systems with parameter-dependent inputs, and applies it to analyze feedback in multi-user Gaussian channels.

Contribution

It introduces a generalized expression for the MI gradient that accounts for parameter-dependent inputs and applies it to feedback in K-user Gaussian MACs, extending the I-MMSE relationship.

Findings

Decomposition of MI gradient into noise, interference, and feedback effects.

Extension of I-MMSE relationship to K-user Gaussian MAC with feedback.

Feedback mitigates interference in symmetric Gaussian MACs.

Abstract

Taking a functional approach, we derive a general expression for the gradient of the Mutual Information (MI) with respect to the system parameters in the stochastic systems. This expression covers the cases in which the system input depends on the system parameters. As an application, we consider the K-user Multiple Access Channels (MAC) with feedback and utilize the obtained results to explore the behavior of these systems in terms of the MI. Specializing the results to the additive Gaussian noise MAC, we extend the MI and Minimum Mean Square Error (MMSE) relationship, i.e., I-MMSE to the K-user Gaussian MAC with feedback. In this derivation, we show that the gradient of MI can be decomposed into three distinct parts, where the first part is the MMSE term originated from noise, and the second and third parts reflect the effects of the interference and feedback, respectively. Then, considering the capacity achieving Fourier-Modulated Estimate Correction (F-MEC) strategy of Kramer, we show how feedback compensates the destructive effects of the users' interference in the K-user symmetric Gaussian MAC.