Verification of indefinite-horizon POMDPs

TL;DR

This paper introduces an abstraction-refinement framework for verifying indefinite-horizon POMDPs, improving scalability by extending the Lovejoy-approach to handle policies based on observation histories.

Contribution

It extends the Lovejoy-approach with an abstraction-refinement framework specifically for POMDP verification, enhancing scalability and practical applicability.

Findings

Significant scalability improvements demonstrated in experiments

Framework effectively handles policies based on observation histories

Extends existing verification methods to more complex POMDPs

Abstract

The verification problem in MDPs asks whether, for any policy resolving the nondeterminism, the probability that something bad happens is bounded by some given threshold. This verification problem is often overly pessimistic, as the policies it considers may depend on the complete system state. This paper considers the verification problem for partially observable MDPs, in which the policies make their decisions based on (the history of) the observations emitted by the system. We present an abstraction-refinement framework extending previous instantiations of the Lovejoy-approach. Our experiments show that this framework significantly improves the scalability of the approach.