Computational Techniques for Reachability Analysis of Partially Observable Discrete Time Stochastic Hybrid Systems

TL;DR

This paper introduces the first computational approach for reachability analysis of partially observable discrete-time stochastic hybrid systems, extending POMDP techniques to handle incomplete measurements for safety verification.

Contribution

It proves the value function is piecewise-linear and convex, enabling the adaptation of point-based value iteration methods to this new class of problems.

Findings

First numerical results for reachability under partial observability.

Extension of POMDP value iteration techniques to hybrid systems.

Application demonstrated on a temperature regulation benchmark.

Abstract

Reachability analysis of hybrid systems has been used as a safety verification tool to assess offline whether the state of a system is capable of remaining within a designated safe region for a given time horizon. Although it has been applied to stochastic hybrid systems, little work has been done on the equally important problem of reachability under incomplete or noisy measurements of the state. Further, there are currently no computational methods or results for reachability analysis of partially observable discrete time stochastic hybrid systems. We provide the first numerical results for solving this problem, by drawing upon existing literature on continuous state partially observable Markov decision processes (POMDPs). We first prove that the value function for the reachability problem (with a multiplicative cost structure) is piecewise-linear and convex, just as for discrete state POMDPs with an additive cost function. Because of these properties, we are able to extend existing point-based value iteration techniques to the reachability problem, demonstrating its applicability on a benchmark temperature regulation problem.