Integrating Explanations in Learning LTL Specifications from Demonstrations

TL;DR

This paper proposes a hybrid approach combining Large Language Models and optimization methods to improve learning Linear Temporal Logic specifications from demonstrations and human explanations, addressing limitations of each method.

Contribution

It introduces a novel method and a tool called Janaka that integrate LLMs and optimization to translate explanations into formal LTL specifications.

Findings

Combining explanations with demonstrations improves LTL learning accuracy.

Janaka effectively translates human explanations into formal specifications.

Hybrid approach outperforms standalone methods in case studies.

Abstract

This paper investigates whether recent advances in Large Language Models (LLMs) can assist in translating human explanations into a format that can robustly support learning Linear Temporal Logic (LTL) from demonstrations. Both LLMs and optimization-based methods can extract LTL specifications from demonstrations; however, they have distinct limitations. LLMs can quickly generate solutions and incorporate human explanations, but their lack of consistency and reliability hampers their applicability in safety-critical domains. On the other hand, optimization-based methods do provide formal guarantees but cannot process natural language explanations and face scalability challenges. We present a principled approach to combining LLMs and optimization-based methods to faithfully translate human explanations and demonstrations into LTL specifications. We have implemented a tool called Janaka based on our approach. Our experiments demonstrate the effectiveness of combining explanations with demonstrations in learning LTL specifications through several case studies.