LIPO+: Frugal Global Optimization for Lipschitz Functions

TL;DR

This paper introduces LIPO+ and AdaLIPO+ algorithms, which are improved versions of existing Lipschitz optimization methods, demonstrating faster convergence and highlighting limitations in high-dimensional spaces.

Contribution

The paper presents simple empirical enhancements to LIPO algorithms, providing faster convergence and formal analysis of their limitations in high dimensions.

Findings

LIPO+ and AdaLIPO+ outperform vanilla algorithms on benchmark functions.

LIPO algorithms are highly susceptible to the curse of dimensionality.

The curse of dimensionality causes LIPO to revert to random search in high dimensions.

Abstract

In this paper, we propose simple yet effective empirical improvements to the algorithms of the LIPO family, introduced in [Malherbe2017], that we call LIPO+ and AdaLIPO+. We compare our methods to the vanilla versions of the algorithms over standard benchmark functions and show that they converge significantly faster. Finally, we show that the LIPO family is very prone to the curse of dimensionality and tends quickly to Pure Random Search when the dimension increases. We give a proof for this, which is also formalized in Lean mathematical language. Source codes and a demo are provided online.