A Hierarchical Reinforcement Learning Method for Persistent Time-Sensitive Tasks

TL;DR

This paper introduces a hierarchical reinforcement learning approach using signal temporal logic and options to effectively learn policies for complex, persistent, and time-sensitive tasks, demonstrated through simulation.

Contribution

It combines STL with the options framework to address persistent and time-sensitive tasks in reinforcement learning, a novel integration for this problem.

Findings

The method learns satisfactory policies with few training cases.

The approach effectively handles complex persistent tasks.

Simulation results validate the method's efficiency.

Abstract

Reinforcement learning has been applied to many interesting problems such as the famous TD-gammon and the inverted helicopter flight. However, little effort has been put into developing methods to learn policies for complex persistent tasks and tasks that are time-sensitive. In this paper, we take a step towards solving this problem by using signal temporal logic (STL) as task specification, and taking advantage of the temporal abstraction feature that the options framework provide. We show via simulation that a relatively easy to implement algorithm that combines STL and options can learn a satisfactory policy with a small number of training cases