Safe Learning and Optimization Techniques: Towards a Survey of the State of the Art

TL;DR

This paper surveys recent advances in safe learning and optimization algorithms across various domains, emphasizing methods that avoid unsafe solutions to prevent irrecoverable losses.

Contribution

It provides a comprehensive review of new safe learning algorithms post-2015, covering reinforcement learning, Gaussian processes, evolutionary algorithms, and active learning, with insights on their connections and future directions.

Findings

Reviewed algorithms from multiple domains including RL and Gaussian processes

Identified key concepts and classifications of safe learning algorithms

Suggested future research directions in safe optimization

Abstract

Safe learning and optimization deals with learning and optimization problems that avoid, as much as possible, the evaluation of non-safe input points, which are solutions, policies, or strategies that cause an irrecoverable loss (e.g., breakage of a machine or equipment, or life threat). Although a comprehensive survey of safe reinforcement learning algorithms was published in 2015, a number of new algorithms have been proposed thereafter, and related works in active learning and in optimization were not considered. This paper reviews those algorithms from a number of domains including reinforcement learning, Gaussian process regression and classification, evolutionary algorithms, and active learning. We provide the fundamental concepts on which the reviewed algorithms are based and a characterization of the individual algorithms. We conclude by explaining how the algorithms are connected and suggestions for future research.