Relative Entropy in Biological Systems

TL;DR

This paper reviews how relative entropy measures the approach to equilibrium in biological systems, linking information theory with thermodynamics and ecology to quantify system dynamics and the Second Law.

Contribution

It synthesizes various theorems showing conditions under which relative entropy decreases, connecting thermodynamic principles with biological and ecological models.

Findings

Relative entropy quantifies distance from equilibrium.

Conditions for nonincreasing relative entropy are established.

Links between thermodynamics and biological information gain are clarified.

Abstract

In this paper we review various information-theoretic characterizations of the approach to equilibrium in biological systems. The replicator equation, evolutionary game theory, Markov processes and chemical reaction networks all describe the dynamics of a population or probability distribution. Under suitable assumptions, the distribution will approach an equilibrium with the passage of time. Relative entropy - that is, the Kullback--Leibler divergence, or various generalizations of this - provides a quantitative measure of how far from equilibrium the system is. We explain various theorems that give conditions under which relative entropy is nonincreasing. In biochemical applications these results can be seen as versions of the Second Law of Thermodynamics, stating that free energy can never increase with the passage of time. In ecological applications, they make precise the notion that a population gains information from its environment as it approaches equilibrium.