Depth-first directional search for nonconvex optimization

TL;DR

This paper introduces a depth-first directional search algorithm for nonconvex optimization that performs thorough line searches along sampled directions, leading to higher accuracy in finding global optima compared to existing methods.

Contribution

The paper presents a novel depth-first directional search (DFDS) algorithm with a geometric convergence analysis for global nonconvex optimization.

Findings

DFDS outperforms existing random search methods in accuracy.

The convergence and complexity of DFDS are rigorously established.

Numerical results show higher success rates in benchmark problems.

Abstract

Random search methods are widely used for global optimization due to their theoretical generality and implementation simplicity. This paper proposes a depth-first directional search (DFDS) algorithm for globally solving nonconvex optimization problems. Motivated by the penetrating beam of a searchlight, DFDS performs a complete stepping line search along each sampled direction before proceeding to the next, contrasting with existing directional search methods that prioritize broad exploratory coverage. We establish the convergence and computational complexity of DFDS through a novel geometric framework that models the success probability of finding a global optimizer as the surface area of a spherical cap. Numerical experiments on benchmark problems demonstrate that DFDS achieves significantly higher accuracy in locating the global optimum compared to other random search methods under the same function evaluation budget.