An Efficient Global Optimization Algorithm with Adaptive Estimates of the Local Lipschitz Constants

TL;DR

This paper introduces HALO, a hyperparameter-free, adaptive global optimization algorithm that uses local Lipschitz estimates to efficiently find global minima, enhanced by coupling with local optimizers.

Contribution

The paper presents HALO, a novel deterministic partition-based global optimization method that adaptively estimates local Lipschitz constants and integrates local search strategies.

Findings

HALO outperforms popular global optimization algorithms on test functions.

HALO is hyperparameter-free and interpretable.

Coupling with local optimizers accelerates convergence.

Abstract

In this work, we present a new deterministic partition-based global optimization algorithm, HALO (Hybrid Adaptive Lipschitzian Optimization), which uses estimates of the local Lipschitz constants associated with different sub-regions of the objective function's domain to compute lower bounds and guide the search toward global minimizers. These estimates are obtained by adaptively balancing the global and local information collected from the algorithm, based on absolute slopes. HALO is hyperparameter-free, eliminating the need for manual tuning, and it highlights the most important variables to help interpret the optimization problem. We also introduce a coupling strategy with local optimization algorithms, both gradient-based and derivative-free, to accelerate convergence. We compare HALO with popular global optimization algorithms on hundreds of test functions. The numerical results are very promising and demonstrate that HALO can expand our arsenal of efficient procedures of efficient procedures for challenging real-world black-box optimization problems. The Python code of HALO is publicly available on GitHub. https://github.com/dannyzx/HALO