A Principle for Global Optimization with Gradients

TL;DR

This paper introduces a principle for using gradient-based non-local quadratic approximants to improve global optimization of differentiable functions with many local minima, analyzing search directions and comparing algorithms.

Contribution

It proposes a new principle for generating search directions from non-local quadratic approximants based on gradients, enhancing global optimization strategies.

Findings

Non-local search directions improve optimization quality

The proposed method outperforms random reinitializations of BFGS

CMA-ES shows competitive performance in experiments

Abstract

This work demonstrates the utility of gradients for the global optimization of certain differentiable functions with many suboptimal local minima. To this end, a principle for generating search directions from non-local quadratic approximants based on gradients of the objective function is analyzed. Experiments measure the quality of non-local search directions as well as the performance of a proposed simplistic algorithm, of the covariance matrix adaptation evolution strategy (CMA-ES), and of a randomly reinitialized Broyden-Fletcher-Goldfarb-Shanno (BFGS) method.