A Framework for Following Temporal Logic Instructions with Unknown Causal Dependencies

TL;DR

This paper introduces a hierarchical reinforcement learning framework that learns symbolic transition models to follow temporal logic instructions with unknown causal dependencies, enabling more efficient multi-task learning in complex environments.

Contribution

It proposes a novel HRL approach using ILP to learn symbolic models, allowing robots to resolve causal dependencies and follow complex temporal logic instructions.

Findings

Outperforms previous methods in discrete and continuous environments

Effectively resolves unknown causal dependencies in complex tasks

Enables efficient multi-task learning with temporal logic instructions

Abstract

Teaching a deep reinforcement learning (RL) agent to follow instructions in multi-task environments is a challenging problem. We consider that user defines every task by a linear temporal logic (LTL) formula. However, some causal dependencies in complex environments may be unknown to the user in advance. Hence, when human user is specifying instructions, the robot cannot solve the tasks by simply following the given instructions. In this work, we propose a hierarchical reinforcement learning (HRL) framework in which a symbolic transition model is learned to efficiently produce high-level plans that can guide the agent efficiently solve different tasks. Specifically, the symbolic transition model is learned by inductive logic programming (ILP) to capture logic rules of state transitions. By planning over the product of the symbolic transition model and the automaton derived from the LTL formula, the agent can resolve causal dependencies and break a causally complex problem down into a sequence of simpler low-level sub-tasks. We evaluate the proposed framework on three environments in both discrete and continuous domains, showing advantages over previous representative methods.