Reachability for Partially Observable Discrete Time Stochastic Hybrid Systems

TL;DR

This paper develops a novel method for stochastic reachability analysis in partially observable discrete-time hybrid systems, maintaining the multiplicative cost structure and providing a dynamic programming solution.

Contribution

It introduces a new approach that preserves the multiplicative cost structure and derives a sufficient statistic for reachability in partially observable stochastic systems.

Findings

Proves the validity of the dynamic programming recursion.

Shows the equivalence to reformulated additive cost problems.

Provides a solution that does not require reformulation.

Abstract

When designing optimal controllers for any system, it is often the case that the true state of the system is unknown to the controller, for example due to noisy measurements or partially observable states. Incomplete state information must be taken into account in the controller's design in order to preserve its optimality. The same is true when performing reachability calculations. To estimate the probability that the state of a stochastic system reaches, or stays within, some set of interest in a given time horizon, it is necessary to find a controller (or at least prove one exists) that drives the system to that set with maximum probability. This controller, however, does not have access to the true state of the system. To date, little work has been done on stochastic reachability calculations with partially observable states. What work has been done relies on converting the reachability optimization problem to one with an additive cost function, for which theoretical results are well known. Our approach is to preserve the multiplicative cost structure when deriving a sufficient statistic that reduces the problem to one of perfect state information. Our transformation includes a change of measure that simplifies the distribution of the sufficient statistic conditioned on its previous value. We develop a dynamic programming recursion for the solution of the equivalent perfect information problem, proving that the recursion is valid, an optimal solution exists, and results in the same solution as to the original problem. We also show that our results are equivalent to those for the reformulated additive cost problem, and so such a reformulation is not required.