Quantifying and managing uncertainty in piecewise-deterministic Markov processes

TL;DR

This paper develops a numerical approach to quantify and control the uncertainty in cumulative costs of piecewise-deterministic Markov processes by solving hyperbolic PDEs, with applications in trajectory planning and environmental management.

Contribution

It introduces a PDE-based framework to compute and bound the distribution of cumulative costs in PDMPs, including control optimization under uncertainty.

Findings

Effective numerical method for CDF computation in PDMPs.

Bounded CDFs provide reliable risk assessments.

Application to fish harvesting demonstrates practical utility.

Abstract

In piecewise-deterministic Markov processes (PDMPs) the state of a finite-dimensional system evolves continuously, but the evolutive equation may change randomly as a result of discrete switches. A running cost is integrated along the corresponding piecewise-deterministic trajectory up to the termination to produce the cumulative cost of the process. We address three natural questions related to uncertainty in cumulative cost of PDMP models: (1) how to compute the Cumulative Distribution Function (CDF) of the cumulative cost when the switching rates are fully known; (2) how to accurately bound the CDF when the switching rates are uncertain; and (3) assuming the PDMP is controlled, how to select a control to optimize that CDF. In all three cases, our approach requires posing a system of suitable hyperbolic partial differential equations, which are then solved numerically on an augmented state space. We illustrate our method using simple examples of trajectory planning under uncertainty for several 1D and 2D first-exit time problems. In the Appendix, we also apply this method to a model of fish harvesting in an environment with random switches in carrying capacity.