Essential barrier height and a probabilistic approach in characterizing potential landscape

TL;DR

This paper introduces a probabilistic method to analyze potential landscapes, revealing how the essential barrier height influences the tail behavior of coupling times in multi-dimensional systems, with applications to neural networks and particle systems.

Contribution

It presents a novel probabilistic framework and the concept of essential barrier height to characterize non-convex potential functions and their landscape properties.

Findings

Coupling times have different tail behaviors for single-well and multi-well potentials.

The negative tail exponent for multi-well potentials decreases exponentially with vanishing noise.

Numerical examples validate the theoretical predictions and assumptions.

Abstract

This paper proposes a probabilistic approach to investigate the shape of landscapes of multi-dimensional potential functions. Under a suitable coupling scheme, two copies of the overdamped Langevin dynamics associated with the potential function are coupled, and the coupling times are collected. Assuming a set of intuitive yet technically challenging conditions on the coupling scheme, it is shown that the tail distributions of the coupling times exhibit qualitatively different dependencies on the noise magnitude for single-well versus multi-well potential functions. More specifically, for convex single-well potentials, the negative tail exponent of the coupling time distribution is uniformly bounded away from zero by the convexity parameter and is independent of the noise magnitude. In contrast, for multi-well potentials, the negative tail exponent decreases exponentially as the noise vanishes, with the decay rate governed by the essential barrier height, a quantity introduced in this paper to characterize the non-convex nature of the potential function. Numerical investigations are conducted for a variety of examples, including the Rosenbrock function, interacting particle systems, and loss functions arising in artificial neural networks. These examples not only illustrate the theoretical results in various contexts but also provide crucial numerical validation of the conjectured assumptions, which are essential to the theoretical analysis yet lie beyond the reach of standard technical tools.