Controller Synthesis for Omega-Regular and Steady-State Specifications

TL;DR

This paper introduces an algorithm for synthesizing controllers for Markov decision processes that satisfy both omega-regular specifications and steady-state constraints, addressing a generalized problem in system control.

Contribution

It proposes a novel method to compute deterministic policies satisfying combined omega-regular and steady-state constraints via integer linear programming.

Findings

Algorithm successfully synthesizes controllers meeting specifications.

Experimental results demonstrate effectiveness and practicality.

Approach extends existing methods to more complex specifications.

Abstract

Given a Markov decision process (MDP) and a linear-time ($\omega$-regular or LTL) specification, the controller synthesis problem aims to compute the optimal policy that satisfies the specification. More recently, problems that reason over the asymptotic behavior of systems have been proposed through the lens of steady-state planning. This entails finding a control policy for an MDP such that the Markov chain induced by the solution policy satisfies a given set of constraints on its steady-state distribution. This paper studies a generalization of the controller synthesis problem for a linear-time specification under steady-state constraints on the asymptotic behavior. We present an algorithm to find a deterministic policy satisfying $\omega$-regular and steady-state constraints by characterizing the solutions as an integer linear program, and experimentally evaluate our approach.