High-dimensional random landscapes: from typical to large deviations

TL;DR

This paper explores the geometric properties of high-dimensional random landscapes, particularly in inference problems like low-rank matrix and tensor estimation, and connects these to large deviation theory.

Contribution

It introduces a framework for analyzing the geometry of high-dimensional landscapes in inference problems and links these to large deviation principles.

Findings

Distinct geometrical features in matrix and tensor landscapes

Methods for characterizing typical landscape realizations

Connections between landscape optimization and large deviation theory

Abstract

In these notes we discuss tools and concepts that emerge when studying high-dimensional random landscapes, i.e., random functions on high-dimensional spaces. As an illustrative example, we consider an inference problem in two forms: low-rank matrix estimation (Case 1) and low-rank tensor estimation (Case 2). We show how to map the inference problem onto the optimization problem of a high-dimensional landscape, which exhibits distinct geometrical properties in the two cases. We discuss methods for characterizing typical realizations of these landscapes and their optimization through local dynamics. We conclude by highlighting connections between the landscape problem and Large Deviation Theory.