Effects of Noise on Entropy Evolution

TL;DR

This paper investigates how the conditional entropy in stochastic systems converges to zero, establishing stability as key, and demonstrates that the convergence rate is exponential and independent of noise amplitude in various models.

Contribution

It provides general conditions for entropy convergence, explicit convergence rate calculations, and extends results to systems with colored and dichotomous noise.

Findings

Conditional entropy converges monotonically and exponentially to zero.

Asymptotic stability is necessary and sufficient for entropy convergence.

Convergence rate is independent of noise amplitude in studied models.

Abstract

We study the convergence properties of the conditional (Kullback-Leibler) entropy in stochastic systems. We have proved very general results showing that asymptotic stability is a necessary and sufficient condition for the monotone convergence of the conditional entropy to its maximal value of zero. Additionally we have made specific calculations of the rate of convergence of this entropy to zero in a one-dimensional situations, illustrated by Ornstein-Uhlenbeck and Rayleigh processes, higher dimensional situations, and a two dimensional Ornstein-Uhlenbeck process with a stochastically perturbed harmonic oscillator and colored noise as examples. We also apply our general results to the problem of conditional entropy convergence in the presence of dichotomous noise. In both the single dimensional and multidimensional cases we are to show that the convergence of the conditional entropy to zero is monotone and at least exponential. In the specific cases of the Ornstein-Uhlenbeck and Rayleigh processes as well as the stochastically perturbed harmonic oscillator and colored noise examples, we have the rather surprising result that the rate of convergence of the entropy to zero is independent of the noise amplitude.