Geometry and universal scaling of Pareto-optimal signal compression

TL;DR

This paper explores the optimal lossy compression of stochastic signals into discrete representations, revealing universal scaling laws and trade-offs in information retention, with implications for efficient signal encoding.

Contribution

It introduces a universal cube-root scaling law for optimal threshold density and demonstrates that non-Gaussian fluctuations can outperform Gaussian models in information-cost trade-offs.

Findings

Optimal threshold density follows a universal cube-root scaling.

Non-Gaussian fluctuations can yield better information-cost trade-offs.

First-order transitions occur in the optimal encoding protocols.

Abstract

I investigate the generic problem of lossy compression of a fluctuating stochastic signal $X$ into a discrete representation $Z$ through optimal thresholding. The signal modulates transition rates of a two-state system described by a binary variable $Y$. Optimising the retained mutual information between $Z$ and $Y$ under a constraint on fixed encoding cost of $Z$ reveals Pareto-optimal trade-offs, determined numerically using genetic algorithms. In the small-noise regime, these fronts are either concave or exhibit piecewise convex ``intrusions'' separated by first-order transitions in the optimal protocol. An analytical high-rate expansion shows that the optimal threshold density follows a universal cube-root scaling with the product of the prior distribution and the Fisher information associated with the response, which holds qualitatively even for few discrete states. Extending the analysis to non-Gaussian fluctuations reveals that for some parameters optimal encoders can yield strictly better information-cost trade-offs than Gaussian surrogates, meaning the same information content can often be achieved with fewer discrete readout states.