Temporal-Logic-Based Intermittent, Optimal, and Safe Continuous-Time Learning for Trajectory Tracking

TL;DR

This paper introduces a safe reinforcement learning framework for trajectory tracking that decomposes complex missions into sub-problems, employs barrier functions for safety, and uses actor-critic algorithms for optimal control, demonstrated through simulations.

Contribution

It presents a novel approach combining temporal logic, barrier functions, and intermittent learning for safe, resource-efficient trajectory tracking in complex missions.

Findings

Effective safety enforcement via barrier functions.

Reduced communication and computation through intermittent control updates.

Successful simulation validation of the proposed framework.

Abstract

In this paper, we develop safe reinforcement-learning-based controllers for systems tasked with accomplishing complex missions that can be expressed as linear temporal logic specifications, similar to those required by search-and-rescue missions. We decompose the original mission into a sequence of tracking sub-problems under safety constraints. We impose the safety conditions by utilizing barrier functions to map the constrained optimal tracking problem in the physical space to an unconstrained one in the transformed space. Furthermore, we develop policies that intermittently update the control signal to solve the tracking sub-problems with reduced burden in the communication and computation resources. Subsequently, an actor-critic algorithm is utilized to solve the underlying Hamilton-Jacobi-Bellman equations. Finally, we support our proposed framework with stability proofs and showcase its efficacy via simulation results.