Conditioning the complexity of random landscapes on marginal optima

TL;DR

This paper introduces a technique to analyze the distribution of marginal optima in complex random landscapes, revealing their characteristics and rarity across different models.

Contribution

A novel method for conditioning the statistics of stationary points on marginality in diverse random landscape models.

Findings

Fully characterized the distribution of marginal optima in various models

Identified conditions under which marginal optima are in the minority

Applied the technique to Gaussian and non-Gaussian landscapes

Abstract

Marginal optima are minima or maxima of a function with many nearly flat directions. In settings with many competing optima, marginal ones tend to attract algorithms and physical dynamics. Often, the important family of marginal attractors are a vanishing minority compared with nonmarginal optima and other unstable stationary points. We introduce a generic technique for conditioning the statistics of stationary points in random landscapes on their marginality, and apply it in three isotropic settings with qualitatively different structure: in the spherical spin-glasses, where the energy is Gaussian and its Hessian is GOE; in multispherical spin glasses, which are Gaussian but non-GOE; and in sums of squared spherical random functions, which are non-Gaussian. In these problems we are able to fully characterize the distribution of marginal optima in the landscape, including when they are in the minority.