LTLf Synthesis on Probabilistic Systems

TL;DR

This paper introduces two algorithms for synthesizing policies in Markov Decision Processes based on finite-trace Linear Temporal Logic (LTLf), addressing a gap in existing tools for finite-horizon properties.

Contribution

It presents the first algorithms for LTLf-based policy synthesis on MDPs, including a reduction approach and a native LTLf method, with a scalability comparison.

Findings

Native LTLf approach outperforms automaton-based methods in scalability.

Two algorithms successfully synthesize policies for finite-trace properties.

The native approach is more scalable than existing automaton generation tools.

Abstract

Many systems are naturally modeled as Markov Decision Processes (MDPs), combining probabilities and strategic actions. Given a model of a system as an MDP and some logical specification of system behavior, the goal of synthesis is to find a policy that maximizes the probability of achieving this behavior. A popular choice for defining behaviors is Linear Temporal Logic (LTL). Policy synthesis on MDPs for properties specified in LTL has been well studied. LTL, however, is defined over infinite traces, while many properties of interest are inherently finite. Linear Temporal Logic over finite traces (LTLf) has been used to express such properties, but no tools exist to solve policy synthesis for MDP behaviors given finite-trace properties. We present two algorithms for solving this synthesis problem: the first via reduction of LTLf to LTL and the second using native tools for LTLf. We compare the scalability of these two approaches for synthesis and show that the native approach offers better scalability compared to existing automaton generation tools for LTL.