ILCL: Inverse Logic-Constraint Learning from Temporally Constrained Demonstrations

TL;DR

This paper introduces ILCL, a novel method for learning temporal logic constraints from demonstrations by combining genetic algorithms and reinforcement learning, enabling effective transfer to real-world tasks.

Contribution

The paper presents a new inverse logic-constraint learning framework that efficiently constructs temporal logic specifications and learns policies under these constraints.

Findings

ILCL outperforms state-of-the-art baselines in benchmark tasks.

ILCL successfully transfers learned constraints to real-world applications.

The method effectively handles the combinatorial complexity of temporal logic learning.

Abstract

We aim to solve the problem of temporal-constraint learning from demonstrations to reproduce demonstration-like logic-constrained behaviors. Learning logic constraints is challenging due to the combinatorially large space of possible specifications and the ill-posed nature of non-Markovian constraints. To figure it out, we introduce a novel temporal-constraint learning method, which we call inverse logic-constraint learning (ILCL). Our method frames ICL as a two-player zero-sum game between 1) a genetic algorithm-based temporal-logic mining (GA-TL-Mining) and 2) logic-constrained reinforcement learning (Logic-CRL). GA-TL-Mining efficiently constructs syntax trees for parameterized truncated linear temporal logic (TLTL) without predefined templates. Subsequently, Logic-CRL finds a policy that maximizes task rewards under the constructed TLTL constraints via a novel constraint redistribution scheme. Our evaluations show ILCL outperforms state-of-the-art baselines in learning and transferring TL constraints on four temporally constrained tasks. We also demonstrate successful transfer to real-world peg-in-shallow-hole tasks.