Optimization of random high-dimensional functions: Structure and algorithms

TL;DR

This paper reviews recent advances in understanding the structure of near-optimal solutions in high-dimensional spin glass models, highlighting the ultrametric organization, critical points, and optimization algorithms.

Contribution

It provides a comprehensive survey of rigorous results on the ultrametric structure and algorithms for mixed p-spin models, advancing the theoretical understanding of high-dimensional optimization landscapes.

Findings

Ultrametric structure of near optima established

Construction of algorithms exploiting ultrametric organization

Rigorous analysis of critical points in spin glass models

Abstract

Replica symmetry breaking postulates that near optima of spin glass Hamiltonians have an ultrametric structure. Namely, near optima can be associated to leaves of a tree, and the Euclidean distance between them corresponds to the distance along this tree. We survey recent progress towards a rigorous proof of this picture in the context of mixed $p$-spin spin glass models. We focus in particular on the following topics: $(i)$~The structure of critical points of the Hamiltonian; $(ii)$~The realization of the ultrametric tree as near optima of a suitable TAP free energy; $(iii)$~The construction of efficient optimization algorithm that exploits this picture.