Automata Learning of Preferences over Temporal Logic Formulas from Pairwise Comparisons

TL;DR

This paper introduces a method to learn user preferences over temporal sequences by modeling them with Preference Deterministic Finite Automata, enabling the inference of complex temporal goals from pairwise comparisons.

Contribution

It formalizes preference inference over temporal goals using PDFA and provides an algorithm to learn minimal PDFA from characteristic samples, addressing computational challenges.

Findings

Proposes PDFA as a model for preferences over temporal sequences.

Develops an algorithm guaranteeing minimal PDFA learning from characteristic samples.

Demonstrates the approach with a robotic motion planning example.

Abstract

Many preference elicitation algorithms consider preference over propositional logic formulas or items with different attributes. In sequential decision making, a user's preference can be a preorder over possible outcomes, each of which is a temporal sequence of events. This paper considers a class of preference inference problems where the user's unknown preference is represented by a preorder over regular languages (sets of temporal sequences), referred to as temporal goals. Given a finite set of pairwise comparisons between finite words, the objective is to learn both the set of temporal goals and the preorder over these goals. We first show that a preference relation over temporal goals can be modeled by a Preference Deterministic Finite Automaton (PDFA), which is a deterministic finite automaton augmented with a preorder over acceptance conditions. The problem of preference inference reduces to learning the PDFA. This problem is shown to be computationally challenging, with the problem of determining whether there exists a PDFA of size smaller than a given integer $k$, consistent with the sample, being NP-Complete. We formalize the properties of characteristic samples and develop an algorithm that guarantees to learn, given a characteristic sample, the minimal PDFA equivalent to the true PDFA from which the sample is drawn. We present the method through a running example and provide detailed analysis using a robotic motion planning problem.