Are Evolutionary Algorithms Safe Optimizers?

TL;DR

This paper formalizes safe optimization problems, compares evolutionary algorithms with machine learning methods, and provides benchmarks and an open-source framework to advance research in safe optimization.

Contribution

It introduces a formal definition of SafeOPs, benchmarks EC algorithms against ML safe optimization methods, and offers an open-source framework for future research.

Findings

EC algorithms' performance varies with SafeOP parameters

Benchmarking reveals strengths and weaknesses of different algorithms

Open-source framework facilitates replication and extension

Abstract

We consider a type of constrained optimization problem, where the violation of a constraint leads to an irrevocable loss, such as breakage of a valuable experimental resource/platform or loss of human life. Such problems are referred to as safe optimization problems (SafeOPs). While SafeOPs have received attention in the machine learning community in recent years, there was little interest in the evolutionary computation (EC) community despite some early attempts between 2009 and 2011. Moreover, there is a lack of acceptable guidelines on how to benchmark different algorithms for SafeOPs, an area where the EC community has significant experience in. Driven by the need for more efficient algorithms and benchmark guidelines for SafeOPs, the objective of this paper is to reignite the interest of this problem class in the EC community. To achieve this we (i) provide a formal definition of SafeOPs and contrast it to other types of optimization problems that the EC community is familiar with, (ii) investigate the impact of key SafeOP parameters on the performance of selected safe optimization algorithms, (iii) benchmark EC against state-of-the-art safe optimization algorithms from the machine learning community, and (iv) provide an open-source Python framework to replicate and extend our work.