Temporal Logic Guided Safe Reinforcement Learning Using Control Barrier Functions

TL;DR

This paper presents a framework that combines temporal logic, control Lyapunov functions, and control barrier functions to enable safe, flexible, and efficient reinforcement learning for complex control tasks with safety guarantees.

Contribution

It introduces a novel integration of temporal logic with control barrier and Lyapunov functions for safe reinforcement learning, accommodating both known and unknown dynamics.

Findings

Enhanced safety during exploration and deployment

Flexible task specification at multiple levels

Effective handling of unknown environmental dynamics

Abstract

Using reinforcement learning to learn control policies is a challenge when the task is complex with potentially long horizons. Ensuring adequate but safe exploration is also crucial for controlling physical systems. In this paper, we use temporal logic to facilitate specification and learning of complex tasks. We combine temporal logic with control Lyapunov functions to improve exploration. We incorporate control barrier functions to safeguard the exploration and deployment process. We develop a flexible and learnable system that allows users to specify task objectives and constraints in different forms and at various levels. The framework is also able to take advantage of known system dynamics and handle unknown environmental dynamics by integrating model-free learning with model-based planning.