SQG-Differential Evolution for difficult optimization problems under a tight function evaluation budget

TL;DR

This paper introduces a hybrid Differential Evolution algorithm incorporating Stochastic Quasi-Gradient methods, designed to efficiently solve complex, high-dimensional, and multi-modal engineering optimization problems with limited function evaluations.

Contribution

A novel derivative-free hybrid DE algorithm combining SQG techniques, showing superior performance on challenging benchmark functions under strict evaluation budgets.

Findings

Performs excellently on high-dimensional, multi-modal test functions.

Computationally inexpensive mutation scheme.

Easily integrable into existing optimization frameworks.

Abstract

In the context of industrial engineering, it is important to integrate efficient computational optimization methods in the product development process. Some of the most challenging simulation-based engineering design optimization problems are characterized by: a large number of design variables, the absence of analytical gradients, highly non-linear objectives and a limited function evaluation budget. Although a huge variety of different optimization algorithms is available, the development and selection of efficient algorithms for problems with these industrial relevant characteristics, remains a challenge. In this communication, a hybrid variant of Differential Evolution (DE) is introduced which combines aspects of Stochastic Quasi-Gradient (SQG) methods within the framework of DE, in order to improve optimization efficiency on problems with the previously mentioned characteristics. The performance of the resulting derivative-free algorithm is compared with other state-of-the-art DE variants on 25 commonly used benchmark functions, under tight function evaluation budget constraints of 1000 evaluations. The experimental results indicate that the new algorithm performs excellent on the 'difficult' (high dimensional, multi-modal, inseparable) test functions. The operations used in the proposed mutation scheme, are computationally inexpensive, and can be easily implemented in existing differential evolution variants or other population-based optimization algorithms by a few lines of program code as an non-invasive optional setting. Besides the applicability of the presented algorithm by itself, the described concepts can serve as a useful and interesting addition to the algorithmic operators in the frameworks of heuristics and evolutionary optimization and computing.