Another Look at the Physics of Large Deviations With Application to Rate-Distortion Theory

TL;DR

This paper offers a new physical interpretation of large deviations rate-functions, viewing them as mechanical equilibrium between systems, leading to novel representations of rate-distortion and channel capacity in information theory.

Contribution

It introduces a mechanical equilibrium perspective for large deviations, extending previous thermal interpretations, and proposes new representations of rate-distortion and channel capacity.

Findings

Mechanical equilibrium interpretation simplifies understanding of rate-functions.

New representations of rate-distortion and channel capacity are developed.

Physical analogy enhances conceptual clarity in information theory.

Abstract

We revisit and extend the physical interpretation recently given to a certain identity between large--deviations rate--functions (as well as applications of this identity to Information Theory), as an instance of thermal equilibrium between several physical systems that are brought into contact. Our new interpretation, of mechanical equilibrium between these systems, is shown to have several advantages relative to that of thermal equilibrium. This physical point of view also provides a trigger to the development of certain alternative representations of the rate--distortion function and channel capacity, which are new to the best knowledge of the author.