An identity of Chernoff bounds with an interpretation in statistical physics and applications in information theory

TL;DR

This paper establishes a fundamental identity between two Chernoff bounds, interprets it through statistical physics, and applies it to information theory, revealing deep connections between these fields.

Contribution

It introduces a novel identity linking Chernoff bounds with statistical physics, providing new insights and applications in information theory.

Findings

Identity between two Chernoff bounds established

Interpretation as isothermal equilibrium in statistical physics

Applications demonstrating the link between information theory and physics

Abstract

An identity between two versions of the Chernoff bound on the probability a certain large deviations event, is established. This identity has an interpretation in statistical physics, namely, an isothermal equilibrium of a composite system that consists of multiple subsystems of particles. Several information--theoretic application examples, where the analysis of this large deviations probability naturally arises, are then described from the viewpoint of this statistical mechanical interpretation. This results in several relationships between information theory and statistical physics, which we hope, the reader will find insightful.