Disordered Systems Insights on Computational Hardness

TL;DR

This review explores how concepts from disordered physics, like phase transitions and replica symmetry breaking, relate to computational hardness in inference and optimization problems, highlighting emerging theoretical evidence for problem difficulty.

Contribution

It introduces models from physics to analyze computational hardness, connecting phase transitions with algorithmic failure and discussing rigorous proof techniques like the overlap gap property and sum-of-squares hierarchy.

Findings

Identification of phase transition points where problems become computationally hard.

Rigorous evidence showing many algorithms fail in the hard regime.

Connections between physics concepts and computational complexity.

Abstract

In this review article, we discuss connections between the physics of disordered systems, phase transitions in inference problems, and computational hardness. We introduce two models representing the behavior of glassy systems, the spiked tensor model and the generalized linear model. We discuss the random (non-planted) versions of these problems as prototypical optimization problems, as well as the planted versions (with a hidden solution) as prototypical problems in statistical inference and learning. Based on ideas from physics, many of these problems have transitions where they are believed to jump from easy (solvable in polynomial time) to hard (requiring exponential time). We discuss several emerging ideas in theoretical computer science and statistics that provide rigorous evidence for hardness by proving that large classes of algorithms fail in the conjectured hard regime. This includes the overlap gap property, a particular mathematization of clustering or dynamical symmetry-breaking, which can be used to show that many algorithms that are local or robust to changes in their input fail. We also discuss the sum-of-squares hierarchy, which places bounds on proofs or algorithms that use low-degree polynomials such as standard spectral methods and semidefinite relaxations, including the Sherrington-Kirkpatrick model. Throughout the manuscript, we present connections to the physics of disordered systems and associated replica symmetry breaking properties.