Adjacency Criterion For Gradient Flow With Multiple Local Maxima

TL;DR

This paper develops an adjacency criterion for gradient flows with multiple local maxima, characterizes equilibria, and applies it to a stochastic double bracket flow to better understand complex optimization landscapes.

Contribution

It introduces a necessary and sufficient adjacency criterion for regions of attraction in gradient flows with multiple maxima and applies it to a specific flow with factorially many maxima.

Findings

Characterized the set of equilibria and their adjacency in complex gradient flows.

Computed the index of critical manifolds and identified neighboring regions.

Applied the criterion to a stochastic flow, providing insights into simulated annealing.

Abstract

In this paper, we investigate the geometry of a general class of gradient flows with multiple local maxima. we decompose the underlying space into disjoint regions of attraction and establish the adjacency criterion. The criterion states a necessary and sufficient condition for two regions of attraction of stable equilibria to be adjacent. We then apply this criterion on a specific type of gradient flow which has as many as n! local maxima. In particular, we characterize the set of equilibria, compute the index of each critical manifold and moreover, find all pairs of adjacent neighbors. As an application of the adjacency criterion, we introduce a stochastic version of the double bracket flow and set up a Markov model to approximate the sample path behavior. The study of this specific prototype with its special structure provides insight into many other difficult problems involving simulated annealing.