Statistical mechanics of lossy data compression using a non-monotonic perceptron

TL;DR

This paper analyzes a lossy data compression scheme using a non-monotonic perceptron, demonstrating it can achieve optimal performance in the limit of large code length through statistical mechanics methods.

Contribution

It introduces a perceptron-based compression scheme inspired by neural network capacity, analytically proving its optimality for large code lengths using the replica method.

Findings

Achieves optimal lossy compression performance asymptotically

Analytical results validated through numerical simulations

Utilizes a non-monotonic perceptron for efficient encoding and decoding

Abstract

The performance of a lossy data compression scheme for uniformly biased Boolean messages is investigated via methods of statistical mechanics. Inspired by a formal similarity to the storage capacity problem in the research of neural networks, we utilize a perceptron of which the transfer function is appropriately designed in order to compress and decode the messages. Employing the replica method, we analytically show that our scheme can achieve the optimal performance known in the framework of lossy compression in most cases when the code length becomes infinity. The validity of the obtained results is numerically confirmed.