LineWalker: Line Search for Black Box Derivative-Free Optimization and Surrogate Model Construction

TL;DR

LineWalker is a derivative-free optimization method that constructs surrogate models along line segments, efficiently identifying extrema and balancing exploration and exploitation without relying on gradient information.

Contribution

It introduces a simple, effective sampling approach using surrogate models and tabu search for black box function optimization along line segments, outperforming existing methods.

Findings

Outperforms Bayesian optimization and NOMAD on nonconvex functions

Effectively identifies local extrema with sparse sampling

Balances exploration and exploitation using tabu search

Abstract

This paper describes a simple, but effective sampling method for optimizing and learning a discrete approximation (or surrogate) of a multi-dimensional function along a one-dimensional line segment of interest. The method does not rely on derivative information and the function to be learned can be a computationally-expensive ``black box'' function that must be queried via simulation or other means. It is assumed that the underlying function is noise-free and smooth, although the algorithm can still be effective when the underlying function is piecewise smooth. The method constructs a smooth surrogate on a set of equally-spaced grid points by evaluating the true function at a sparse set of judiciously chosen grid points. At each iteration, the surrogate's non-tabu local minima and maxima are identified as candidates for sampling. Tabu search constructs are also used to promote diversification. If no non-tabu extrema are identified, a simple exploration step is taken by sampling the midpoint of the largest unexplored interval. The algorithm continues until a user-defined function evaluation limit is reached. Numerous examples are shown to illustrate the algorithm's efficacy and superiority relative to state-of-the-art methods, including Bayesian optimization and NOMAD, on primarily nonconvex test functions.