CMA-ES for Safe Optimization

TL;DR

This paper introduces safe CMA-ES, an evolutionary algorithm designed for safe optimization that effectively minimizes unsafe evaluations while optimizing solutions in safety-critical applications.

Contribution

It proposes a novel safe CMA-ES method that estimates safety function Lipschitz constants and projects samples to safe regions, improving safety and efficiency over existing methods.

Findings

Safe CMA-ES reduces unsafe evaluations effectively.

It outperforms existing methods in numerical simulations.

The method maintains optimization performance while ensuring safety.

Abstract

In several real-world applications in medical and control engineering, there are unsafe solutions whose evaluations involve inherent risk. This optimization setting is known as safe optimization and formulated as a specialized type of constrained optimization problem with constraints for safety functions. Safe optimization requires performing efficient optimization without evaluating unsafe solutions. A few studies have proposed the optimization methods for safe optimization based on Bayesian optimization and the evolutionary algorithm. However, Bayesian optimization-based methods often struggle to achieve superior solutions, and the evolutionary algorithm-based method fails to effectively reduce unsafe evaluations. This study focuses on CMA-ES as an efficient evolutionary algorithm and proposes an optimization method termed safe CMA-ES. The safe CMA-ES is designed to achieve both safety and efficiency in safe optimization. The safe CMA-ES estimates the Lipschitz constants of safety functions transformed with the distribution parameters using the maximum norm of the gradient in Gaussian process regression. Subsequently, the safe CMA-ES projects the samples to the nearest point in the safe region constructed with the estimated Lipschitz constants. The numerical simulation using the benchmark functions shows that the safe CMA-ES successfully performs optimization, suppressing the unsafe evaluations, while the existing methods struggle to significantly reduce the unsafe evaluations.