Characterizing the Topography of Multi-dimensional Energy Landscapes

TL;DR

This paper introduces systematic methods for analyzing the topography of high-dimensional functions using local optimization from random starting points, providing quantitative measures and error estimates relevant to complex problems.

Contribution

It presents generic techniques for characterizing the topography of high-dimensional functions through local optimization and random sampling, with practical measures and error bounds.

Findings

Quantitative measures of function topography are developed.

Methods are effective for functions with hundreds or thousands of variables.

Error estimates for topographic analysis are provided.

Abstract

A basic issue in optimization, inverse theory,neural networks, computational chemistry and many other problems is the geometrical characterization of high dimensional functions. In inverse calculations one aims to characterize the set of models that fit the data (among other constraints). If the data misfit function is unimodal then one can find its peak by local optimization methods and characterize its width (related to the range of data-fitting models) by estimating derivatives at this peak. On the other hand, if there are local extrema, then a number of interesting and difficult problems arise. Are the local extrema important compared to the global or can they be eliminated (e.g., by smoothing) without significant loss of information? Is there a sufficiently small number of local extrema that they can be enumerated via local optimization? What are the basins of attraction of these local extrema? Can two extrema be joined by a path that never goes uphill? Can the whole problem be reduced to one of enumerating the local extrema and their basins of attraction? For locally ill-conditioned functions, premature convergence of local optimization can be confused with the presence of local extrema. Addressing any of these issues requires topographic information about the functions under study. But in many applications these functions may have hundreds or thousands of variables and can only be evaluated pointwise (by some numerical method for instance). In this paper we describe systematic (but generic) methods of analysing the topography of high dimensional functions using local optimization methods applied to randomly chosen starting models. We provide a number of quantitative measures of function topography that have proven to be useful in practical problems along with error estimates.