Spin Glass Concepts in Computer Science, Statistics, and Learning

TL;DR

This paper explores how spin glass theory provides valuable insights into high-dimensional optimization, statistical learning, and algorithm analysis, bridging physics concepts with computer science and statistics.

Contribution

It explains how spin glass ideas have influenced recent advances in high-dimensional statistics, machine learning, and optimization theory.

Findings

Spin glass theory helps understand the structure of minima in high-dimensional functions.

Near-minima analysis informs the design and analysis of optimization algorithms.

Connections between physics and computer science lead to new theoretical insights.

Abstract

Spin glass theory studies the structure of sublevel sets and minima (or near-minima) of certain classes of random functions in high dimension. Near-minima of random functions also play an important role in high-dimensional statistics and statistical learning, where minimizing the empirical risk (which is a random function of the model parameters) is the method of choice for learning a statistical model from noisy data. Finally, near-minima of random functions are obviously central to average-case analysis of optimization algorithms. Computer science, statistics, and machine learning naturally lead to questions that are traditionally not addressed within physics and mathematical physics. I will try to explain how ideas from spin glass theory have seeded recent developments in these fields. (This article was written on the occasion of the 2024 Abel Prize to Michel Talagrand.)