A Note on the Equivalence of Gibbs Free Energy and Information Theoretic Capacity

TL;DR

This paper demonstrates the fundamental equivalence between Gibbs free energy minimization in thermodynamics and the maximization of information capacity in communication systems, using a simple Ising model as illustration.

Contribution

It reveals that Gibbs free energy and information theoretic capacity are mathematically equivalent, bridging thermodynamics and information theory through a dual use Ising model.

Findings

Gibbs free energy minimization equals mutual information maximization in a dual model.

Detailed calculation for binary symmetric channel shows the equivalence explicitly.

Theoretical connection between thermodynamic and information-theoretic concepts is established.

Abstract

The minimization of Gibbs free energy is based on the changes in work and free energy that occur in a physical or chemical system. The maximization of mutual information, the capacity, of a noisy channel is determined based on the marginal probabilities and conditional entropies associated with a communications system. As different as the procedures might first appear, through the exploration of a simple, "dual use" Ising model, it is seen that the two concepts are in fact the same. In particular, the case of a binary symmetric channel is calculated in detail.