Convexity of mutual information along the heat flow

TL;DR

This paper investigates the convexity properties of mutual information during the heat flow evolution, establishing conditions for convexity and providing counterexamples for nonconvexity at small times.

Contribution

It proves mutual information is convex along heat flow for log-concave initial distributions and eventually convex for broader classes, with counterexamples at small times.

Findings

Mutual information is convex over time for log-concave initial distributions.

Mutual information becomes convex at large times for distributions with finite moments and Fisher information.

Counterexamples show mutual information can be nonconvex at small times.

Abstract

We study the convexity of mutual information along the evolution of the heat equation. We prove that if the initial distribution is log-concave, then mutual information is always a convex function of time. We also prove that if the initial distribution is either bounded, or has finite fourth moment and Fisher information, then mutual information is eventually convex, i.e., convex for all large time. Finally, we provide counterexamples to show that mutual information can be nonconvex at small time.