Shannon versus Kullback-Leibler Entropies in Nonequilibrium Random Motion

TL;DR

This paper compares Shannon and Kullback-Leibler entropies in analyzing nonequilibrium diffusion processes, highlighting their different behaviors and sensitivities to environmental interactions.

Contribution

It provides a detailed comparison of Shannon and Kullback-Leibler entropies in nonequilibrium diffusion, revealing their distinct dynamical properties and implications for understanding entropy production.

Findings

Kullback-Leibler entropy monotonically approaches equilibrium.

Shannon entropy rate can be negative, indicating power transfer.

Kullback-Leibler entropy rate matches Shannon entropy production in Smoluchowski diffusions.

Abstract

We analyze dynamical properties of the Shannon information entropy of a continuous probability distribution, which is driven by a standard diffusion process. This entropy choice is confronted with another option, employing the conditional Kullback-Leibler entropy. Both entropies discriminate among various probability distributions, either statically or in the time domain. An asymptotic approach towards equilibrium is typically monotonic in terms of the Kullback entropy. The Shannon entropy time rate needs not to be positive and is a sensitive indicator of the power transfer processes (removal/supply) due to an active environment. In the case of Smoluchowski diffusions, the Kullback entropy time rate coincides with the Shannon entropy "production" rate.