On the Optimality of Uncertain MDP Abstractions

TL;DR

This paper investigates the asymptotic optimality and completeness of abstraction-based control synthesis for uncertain MDPs in nonlinear stochastic systems, providing theoretical guarantees and conditions for optimality.

Contribution

It introduces a new algorithm that refines UMDP abstractions to achieve near-optimal control with performance guarantees, supported by a theoretical analysis of conditions for asymptotic optimality.

Findings

Set-valued MDP abstractions satisfy the vanishing ambiguity criterion.

Interval MDP abstractions do not guarantee asymptotic optimality.

The proposed algorithm iteratively refines abstractions until near-optimality is achieved.

Abstract

We study the asymptotic optimality of abstraction-based control synthesis algorithms. Specifically, we consider uncertain MDP (UMDP) abstraction, and investigate whether refinement leads to optimal results, i.e., an optimal controller and zero error bound. Additionally, we study completeness of abstraction-refinement algorithms, i.e., that the algorithm produces near-optimal results in finite time. The focus is on nonlinear stochastic systems with general vector fields and temporal logic specifications. We present an algorithm that abstracts the system into a UMDP and synthesizes a controller with performance guarantees via robust dynamic programming. Then, the algorithm iteratively refines the abstraction until a near-optimality criterion is met. A thorough theoretical analysis reveals a sufficient condition, which we denote vanishing ambiguity, guaranteeing asymptotic optimality of the abstraction process and completeness of the algorithm. We show that set-valued MDP abstractions satisfy this criterion, whereas interval MDP abstractions lack such a guarantee.