Higher Order Derivatives in Costa's Entropy Power Inequality

TL;DR

This paper investigates higher order derivatives of the differential entropy in Costa's EPI, revealing new convexity properties of Fisher information and proposing conjectures on the monotonicity of derivatives.

Contribution

It establishes the nonnegativity of the third and fourth derivatives of entropy, introduces conjectures on higher derivatives, and provides geometric insights into covariance-preserving transformations.

Findings

Third derivative of entropy is nonnegative, implying Fisher information is convex.

Fourth derivative of entropy is nonpositive, indicating specific curvature properties.

Proposes conjectures on the sign of higher order derivatives and the convexity of log Fisher information.

Abstract

Let $X$ be an arbitrary continuous random variable and $Z$ be an independent Gaussian random variable with zero mean and unit variance. For $t~>~0$, Costa proved that $e^{2h(X+\sqrt{t}Z)}$ is concave in $t$, where the proof hinged on the first and second order derivatives of $h(X+\sqrt{t}Z)$. Specifically, these two derivatives are signed, i.e., $\frac{\partial}{\partial t}h(X+\sqrt{t}Z) \geq 0$ and $\frac{\partial^2}{\partial t^2}h(X+\sqrt{t}Z) \leq 0$. In this paper, we show that the third order derivative of $h(X+\sqrt{t}Z)$ is nonnegative, which implies that the Fisher information $J(X+\sqrt{t}Z)$ is convex in $t$. We further show that the fourth order derivative of $h(X+\sqrt{t}Z)$ is nonpositive. Following the first four derivatives, we make two conjectures on $h(X+\sqrt{t}Z)$: the first is that $\frac{\partial^n}{\partial t^n} h(X+\sqrt{t}Z)$ is nonnegative in $t$ if $n$ is odd, and nonpositive otherwise; the second is that $\log J(X+\sqrt{t}Z)$ is convex in $t$. The first conjecture can be rephrased in the context of completely monotone functions: $J(X+\sqrt{t}Z)$ is completely monotone in $t$. The history of the first conjecture may date back to a problem in mathematical physics studied by McKean in 1966. Apart from these results, we provide a geometrical interpretation to the covariance-preserving transformation and study the concavity of $h(\sqrt{t}X+\sqrt{1-t}Z)$, revealing its connection with Costa's EPI.