Scalable control synthesis for stochastic systems via structural IMDP abstractions

TL;DR

This paper presents a scalable abstraction framework for synthesizing controllers for nonlinear stochastic systems using orthogonally decoupled Interval Markov Decision Processes, improving memory efficiency and reducing conservatism in probabilistic reach-avoid specifications.

Contribution

It introduces odIMDPs, a new class of robust Markov models, enabling compositional abstractions with lower memory complexity and more accurate control strategies for stochastic systems.

Findings

Outperforms existing methods in memory efficiency.

Reduces conservatism in control strategies.

Demonstrated effectiveness on 7D benchmark systems.

Abstract

This paper introduces a novel abstraction-based framework for controller synthesis of nonlinear discrete-time stochastic systems. The focus is on probabilistic reach-avoid specifications. The framework is based on abstracting a stochastic system into a new class of robust Markov models, called orthogonally decoupled Interval Markov Decision Processes (odIMDPs). Specifically, an odIMDPs is a class of robust Markov processes, where the transition probabilities between each pair of states are uncertain and have the product form. We show that such a specific form in the transition probabilities allows one to build compositional abstractions of stochastic systems that, for each state, are only required to store the marginal probability bounds of the original system. This leads to improved memory complexity for our approach compared to commonly employed abstraction-based approaches. Furthermore, we show that an optimal control strategy for a odIMDPs can be computed by solving a set of linear problems. When the resulting strategy is mapped back to the original system, it is guaranteed to lead to reduced conservatism compared to existing approaches. To test our theoretical framework, we perform an extensive empirical comparison of our methods against Interval Markov Decision Process- and Markov Decision Process-based approaches on various benchmarks including 7D systems. Our empirical analysis shows that our approach substantially outperforms state-of-the-art approaches in terms of both memory requirements and the conservatism of the results.