Safe model-based Reinforcement Learning via Model Predictive Control and Control Barrier Functions

TL;DR

This paper presents a novel safe model-based reinforcement learning framework that integrates MPC and CBF with parameterized class $\\ extbf{K}$ functions, including neural architectures, to learn safe control policies from data.

Contribution

It introduces three variations of a framework combining MPC and CBF with learnable parameters, enhancing safety and performance in control tasks.

Findings

Improved safety and performance in double-integrator experiments.

Effective learning of safe control policies from data.

Framework accommodates neural network parameterizations.

Abstract

Optimal control strategies are often combined with safety certificates to ensure both performance and safety in safety-critical systems. A prominent example is combining Model Predictive Control (MPC) with Control Barrier Functions (CBF). Yet, efficient tuning of MPC parameters and choosing an appropriate class $\mathcal{K}$ function in the CBF is challenging and problem dependent. This paper introduces a safe model-based Reinforcement Learning (RL) framework where a parametric MPC controller incorporates a CBF constraint with a parameterized class $\mathcal{K}$ function and serves as a function approximator to learn improved safe control policies from data. Three variations of the framework are introduced, distinguished by the way the optimization problem is formulated and the class $\mathcal{K}$ function is parameterized, including neural architectures. Numerical experiments on a discrete double-integrator with static and dynamic obstacles demonstrate that the proposed methods improve performance while ensuring safety.