Probabilistic Planning with Preferences over Temporal Goals

TL;DR

This paper introduces a formal language and planning algorithm for optimizing the satisfaction of qualitative preferences over temporal goals in stochastic systems, enabling more nuanced decision-making within finite time constraints.

Contribution

It proposes a novel automata-theoretic preference specification framework and an algorithm for time-constrained probabilistic planning in labeled Markov decision processes.

Findings

Successful implementation in a stochastic gridworld example

Effective maximization of preference satisfaction within time limits

Discussion of potential extensions to the preference model

Abstract

We present a formal language for specifying qualitative preferences over temporal goals and a preference-based planning method in stochastic systems. Using automata-theoretic modeling, the proposed specification allows us to express preferences over different sets of outcomes, where each outcome describes a set of temporal sequences of subgoals. We define the value of preference satisfaction given a stochastic process over possible outcomes and develop an algorithm for time-constrained probabilistic planning in labeled Markov decision processes where an agent aims to maximally satisfy its preference formula within a pre-defined finite time duration. We present experimental results using a stochastic gridworld example and discuss possible extensions of the proposed preference model.