Proximal basin hopping: global optimization with guarantees

TL;DR

Proximal Basin Hopping (PBH) is a new theoretical framework and algorithm for global optimization that guarantees convergence with high probability, outperforming existing methods especially in high-dimensional problems.

Contribution

The paper introduces PBH, a novel framework combining proximal optimization and local minimization, with proven convergence guarantees and superior performance on synthetic and real problems.

Findings

PBH converges to the global minimizer with high probability using finite samples.

PBH outperforms well-known algorithms with theoretical guarantees on synthetic functions.

Performance gap increases with higher dimensions.

Abstract

Global optimization is a challenging problem, with plenty of algorithms displaying empirical success, but scarce theoretical backing. In this work, we propose a new theoretical framework called Proximal Basin Hopping (PBH), carefully tailored to combine proximal optimization and local minimization. We use it to construct a practical algorithm that converges to the global minimizer with high probability, when using a finite amount of samples. Proximal Basin Hopping outperforms well known algorithms with theoretical backing on standard synthetic hard functions, and real problems such as fitting scaling laws for deep learning. Furthermore, the higher the dimension, the better the performance gap.