A statistical-mechanical view on source coding: physical compression and data compression

TL;DR

This paper establishes an analogy between lossy data compression and a physical model of a chain of particles, providing new insights into compression performance through statistical mechanics.

Contribution

It introduces a novel analogy linking source coding to physical models, enabling analysis of lossy compression with memory using statistical mechanics tools.

Findings

Free energy difference relates to rate-distortion function.

Contracting force proportional to the derivative of the rate-distortion function.

Provides a physical perspective on optimal lossy compression schemes.

Abstract

We draw a certain analogy between the classical information-theoretic problem of lossy data compression (source coding) of memoryless information sources and the statistical mechanical behavior of a certain model of a chain of connected particles (e.g., a polymer) that is subjected to a contracting force. The free energy difference pertaining to such a contraction turns out to be proportional to the rate-distortion function in the analogous data compression model, and the contracting force is proportional to the derivative this function. Beyond the fact that this analogy may be interesting on its own right, it may provide a physical perspective on the behavior of optimum schemes for lossy data compression (and perhaps also, an information-theoretic perspective on certain physical system models). Moreover, it triggers the derivation of lossy compression performance for systems with memory, using analysis tools and insights from statistical mechanics.