Adaptive direct search algorithms for constrained optimization

TL;DR

This paper introduces Adaptive Direct Search (ADS), a new class of derivative-free optimization algorithms that combines the strengths of existing methods while avoiding their limitations, with promising computational results.

Contribution

The paper proposes ADS, a novel direct search method using a punctured space acceptance rule, unifying and improving upon MADS and SDDS principles.

Findings

ADS performs competitively with MADS and SDDS in constrained problems.

ADS demonstrates flexible search capabilities without mesh restrictions.

Computational experiments show improved convergence in various settings.

Abstract

Two families of directional direct search methods have emerged in derivative-free and blackbox optimization (DFO and BBO), each based on distinct principles: Mesh Adaptive Direct Search (MADS) and Sufficient Decrease Direct Search (SDDS). MADS restricts trial points to a mesh and accepts any improvement, ensuring none are missed, but at the cost of restraining the placement of trial points. SDDS allows greater freedom by evaluating points anywhere in the space, but accepts only those yielding a sufficient decrease in the objective function value, which may lead to discarding improving points. This work introduces a new class of methods, Adaptive Direct Search (ADS), which uses a novel acceptance rule based on the so-called punctured space, avoiding both meshes and sufficient decrease conditions. ADS enables flexible search while addressing the limitations of MADS and SDDS, and retains the theoretical foundations of directional direct search. Computational results in constrained and unconstrained settings highlight its performance compared to both MADS and SDDS.