The closest vector problem and the zero-temperature p-spin landscape for lossy compression

TL;DR

This paper investigates a high-dimensional constrained optimization problem related to lossy compression, revealing phase transitions and the impact of variable domain size on solution quality through analytical and numerical methods.

Contribution

It provides an analytical solution for a specific case, explores phase transitions in more general ensembles, and extends results to variables over Galois fields, highlighting the effects on optimal solutions.

Findings

Partial analytical solution for variables involved in at most two constraints

Identification of a glassy phase with many local minima

Increasing q in GF(q) improves the optimal solution quality

Abstract

We consider a high-dimensional random constrained optimization problem in which a set of binary variables is subjected to a linear system of equations. The cost function is a simple linear cost, measuring the Hamming distance with respect to a reference configuration. Despite its apparent simplicity, this problem exhibits a rich phenomenology. We show that different situations arise depending on the random ensemble of linear systems. When each variable is involved in at most two linear constraints, we show that the problem can be partially solved analytically, in particular we show that upon convergence, the zero-temperature limit of the cavity equations returns the optimal solution. We then study the geometrical properties of more general random ensembles. In particular we observe a range in the density of constraints at which the systems enters a glassy phase where the cost function has many minima. Interestingly, the algorithmic performances are only sensitive to another phase transition affecting the structure of configurations allowed by the linear constraints. We also extend our results to variables belonging to $\text{GF}(q)$, the Galois Field of order $q$. We show that increasing the value of $q$ allows to achieve a better optimum, which is confirmed by the Replica Symmetric cavity method predictions.