Statistical Mechanical Approach to Error Exponents of Lossy Data Compression

TL;DR

This paper introduces a statistical mechanical method using the replica approach to evaluate error exponents in lossy data compression, extending applicability beyond traditional random code ensembles and demonstrating the optimality of certain perceptron-based codes.

Contribution

It develops a replica method framework for analyzing error exponents in lossy compression, applicable to broader code ensembles and identifying optimal perceptron-based codes.

Findings

Replica method reproduces known optimal error exponents.

Perceptron-based codes can achieve optimal error exponents.

Framework extends analysis to non-random code ensembles.

Abstract

We present herein a scheme by which to accurately evaluate the error exponents of a lossy data compression problem, which characterize average probabilities over a code ensemble of compression failure and success above or below a critical compression rate, respectively, utilizing the replica method (RM). Although the existing method used in information theory (IT) is, in practice, limited to ensembles of randomly constructed codes, the proposed RM-based approach can be applied to a wider class of ensembles. This approach reproduces the optimal expressions of the error exponents achieved by the random code ensembles, which are known in IT. In addition, the proposed framework is used to show that codes composed of non-monotonic perceptrons of a specific type can provide the optimal exponents in most cases, which is supported by numerical experiments.