IT formulae for gamma target: mutual information and relative entropy

TL;DR

This paper develops new mathematical identities and a non-linear channel model for gamma-distributed inputs, deriving a formula for how mutual information changes with channel quality, with implications for information theory.

Contribution

It introduces novel Stein identities and a gamma-specific non-linear channel, providing an explicit formula for the mutual information derivative with respect to channel parameters.

Findings

Derived a simple formula for the derivative of mutual information.

Established bounds and asymptotics for the mutual information.

Linked gamma distribution identities to information-theoretic measures.

Abstract

In this paper, we introduce new Stein identities for gamma target distribution as well as a new non-linear channel specifically designed for gamma inputs. From these two ingredients, we derive an explicit and simple formula for the derivative of the input-output mutual information of this non-linear channel with respect to the channel quality parameter. This relation is reminiscent of the well-known link between the derivative of the input-output mutual information of additive Gaussian noise channel with respect to the signal-to-noise ratio and the minimum mean-square error. The proof relies on a rescaled version of De Bruijn identity for gamma target distribution together with a stochastic representation for the gamma specific Fisher information. Finally, we are able to derive precise bounds and asymptotics for the input-output mutual information of the non-linear channel with gamma inputs.