# A predictive safety filter for learning-based control of constrained nonlinear dynamical systems

## TL;DR

This paper introduces a predictive safety filter that enables reinforcement learning algorithms to operate safely on constrained nonlinear systems by dynamically modifying control inputs based on a data-driven model predictive control approach.

## Contribution

It presents a novel safety filter framework that transforms constrained nonlinear systems into safe systems, compatible with any RL algorithm, using a model predictive control formulation.

## Key findings

- The safety filter effectively ensures safety constraints are met during control.
- The approach is applicable to continuous state and input spaces.
- It integrates seamlessly with existing RL algorithms.

## Abstract

The transfer of reinforcement learning (RL) techniques into real-world applications is challenged by safety requirements in the presence of physical limitations. Most RL methods, in particular the most popular algorithms, do not support explicit consideration of state and input constraints. In this paper, we address this problem for nonlinear systems with continuous state and input spaces by introducing a predictive safety filter, which is able to turn a constrained dynamical system into an unconstrained safe system and to which any RL algorithm can be applied `out-of-the-box'. The predictive safety filter receives the proposed control input and decides, based on the current system state, if it can be safely applied to the real system, or if it has to be modified otherwise. Safety is thereby established by a continuously updated safety policy, which is based on a model predictive control formulation using a data-driven system model and considering state and input dependent uncertainties.