Barrier Certified Safety Learning Control: When Sum-of-Square Programming Meets Reinforcement Learning

TL;DR

This paper introduces a novel safety learning control method combining sum-of-squares programming with reinforcement learning to guarantee safety in control systems, demonstrated on an inverted pendulum model.

Contribution

It proposes a new algorithm that uses sum-of-squares programming to ensure safety in reinforcement learning, addressing safety guarantees in control systems.

Findings

Sum-of-squares programming effectively maintains safety in reinforcement learning.

The proposed method outperforms quadratic programming-based approaches.

Demonstrated success on an inverted pendulum model.

Abstract

Safety guarantee is essential in many engineering implementations. Reinforcement learning provides a useful way to strengthen safety. However, reinforcement learning algorithms cannot completely guarantee safety over realistic operations. To address this issue, this work adopts control barrier functions over reinforcement learning, and proposes a compensated algorithm to completely maintain safety. Specifically, a sum-of-squares programming has been exploited to search for the optimal controller, and tune the learning hyperparameters simultaneously. Thus, the control actions are pledged to be always within the safe region. The effectiveness of proposed method is demonstrated via an inverted pendulum model. Compared to quadratic programming based reinforcement learning methods, our sum-of-squares programming based reinforcement learning has shown its superiority.