Hessian and concavity of mutual information, differential entropy, and entropy power in linear vector Gaussian channels

TL;DR

This paper derives closed-form expressions for the Hessian of mutual information and differential entropy in linear vector Gaussian channels, enabling analysis of their concavity and leading to a multivariate entropy power inequality.

Contribution

It provides new closed-form formulas for the Hessian of key information measures in Gaussian channels, advancing understanding of their concavity properties.

Findings

Hessian expressions for mutual information and differential entropy are derived.

Concavity properties of these information measures are analyzed.

A multivariate entropy power inequality is established.

Abstract

Within the framework of linear vector Gaussian channels with arbitrary signaling, closed-form expressions for the Jacobian of the minimum mean square error and Fisher information matrices with respect to arbitrary parameters of the system are calculated in this paper. Capitalizing on prior research where the minimum mean square error and Fisher information matrices were linked to information-theoretic quantities through differentiation, closed-form expressions for the Hessian of the mutual information and the differential entropy are derived. These expressions are then used to assess the concavity properties of mutual information and differential entropy under different channel conditions and also to derive a multivariate version of the entropy power inequality due to Costa.