CMA-ES with Radial Basis Function Surrogate for Black-Box Optimization

TL;DR

This paper introduces CMA-SAO, a surrogate-assisted CMA-ES algorithm that reduces function evaluations in black-box optimization, especially for costly problems, by integrating Gaussian surrogate models to improve efficiency.

Contribution

The paper presents a novel surrogate-assisted CMA-ES method that adaptively builds surrogate models to enhance optimization efficiency in black-box problems.

Findings

CMA-SAO significantly reduces function evaluations compared to existing algorithms.

The surrogate model integration improves the efficiency of the CMA-ES algorithm.

Empirical results demonstrate enhanced performance in costly optimization scenarios.

Abstract

Evolutionary optimization algorithms often face defects and limitations that complicate the evolution processes or even prevent them from reaching the global optimum. A notable constraint pertains to the considerable quantity of function evaluations required to achieve the intended solution. This concern assumes heightened significance when addressing costly optimization problems. However, recent research has shown that integrating machine learning methods, specifically surrogate models, with evolutionary optimization can enhance various aspects of these algorithms. Among the evolutionary algorithms, the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) is particularly favored. This preference is due to its use of Gaussian distribution for calculating evolution and its ability to adapt optimization parameters, which reduces the need for user intervention in adjusting initial parameters. In this research endeavor, we propose the adoption of surrogate models within the CMA-ES framework called CMA-SAO to develop an initial surrogate model that facilitates the adaptation of optimization parameters through the acquisition of pertinent information derived from the associated surrogate model. Empirical validation reveals that CMA-SAO algorithm markedly diminishes the number of function evaluations in comparison to prevailing algorithms, thereby providing a significant enhancement in operational efficiency.