High-dimensional inference: a statistical mechanics perspective

TL;DR

This paper explores the deep connections between high-dimensional statistical inference and statistical mechanics, illustrating how concepts from physics can inform understanding of complex inference problems in machine learning.

Contribution

It provides an introduction to the statistical mechanics perspective on high-dimensional inference, highlighting key models and their physical analogies.

Findings

Links between inference and statistical physics are elucidated.

Paradigmatic models are described using statistical mechanics language.

The paper serves as an accessible introduction to the field.

Abstract

Statistical inference is the science of drawing conclusions about some system from data. In modern signal processing and machine learning, inference is done in very high dimension: very many unknown characteristics about the system have to be deduced from a lot of high-dimensional noisy data. This "high-dimensional regime" is reminiscent of statistical mechanics, which aims at describing the macroscopic behavior of a complex system based on the knowledge of its microscopic interactions. It is by now clear that there are many connections between inference and statistical physics. This article aims at emphasizing some of the deep links connecting these apparently separated disciplines through the description of paradigmatic models of high-dimensional inference in the language of statistical mechanics. This article has been published in the issue on artificial intelligence of Ithaca, an Italian popularization-of-science journal. The selected topics and references are highly biased and not intended to be exhaustive in any ways. Its purpose is to serve as introduction to statistical mechanics of inference through a very specific angle that corresponds to my own tastes and limited knowledge.