Adaptive Block Coordinate DIRECT algorithm

TL;DR

This paper introduces an adaptive block coordinate version of the DIRECT algorithm that improves efficiency and convergence in bound constrained optimization, especially in high dimensions and flat objective functions.

Contribution

The paper proposes a novel coordinate DIRECT algorithm with adaptive block size and SQP-based local search, enhancing performance over traditional DIRECT methods.

Findings

Achieves better efficiency and accuracy in numerical experiments.

Handles high-dimensional problems more effectively.

Accelerates convergence on flat objective functions.

Abstract

DIviding RECTangles (DIRECT) is an efficient and popular method in dealing with bound constrained optimization problems. However, DIRECT suffers from dimension curse, since its computational complexity soars when dimension increases. Besides, DIRECT also converges slowly when the objective function is flat. In this paper, we propose a coordinate DIRECT algorithm, which coincides with the spirits of other coordinate update algorithms. We transform the original problem into a series of sub-problems, where only one or several coordinates are selected to optimize and the rest keeps fixed. For each sub-problem, coordinately dividing the feasible domain enjoys low computational burden. Besides, we develop adaptive schemes to keep the efficiency and flexibility to tackle different functions. Specifically, we use block coordinate update, of which the size could be adaptively selected, and we also employ sequential quadratic programming (SQP) to conduct the local search to efficiently accelerate the convergence even when the objective function is flat. With these techniques, the proposed algorithm achieves promising performance on both efficiency and accuracy in numerical experiments.