The Stochastic Reach-Avoid Problem and Set Characterization for Diffusions

TL;DR

This paper develops a novel control-theoretic framework for stochastic reach-avoid problems with less strict requirements than almost-sure conditions, using PDE characterizations and viscosity solutions for numerical solutions.

Contribution

It introduces a new methodology connecting stochastic reachability with optimal control, including a weak dynamic programming principle and PDE characterization for discontinuous payoffs.

Findings

Established a connection between reach-avoid and stochastic control problems.

Derived a weak dynamic programming principle for discontinuous payoffs.

Validated the approach on the stochastic Zermelo navigation problem.

Abstract

In this article we approach a class of stochastic reachability problems with state constraints from an optimal control perspective. Preceding approaches to solving these reachability problems are either confined to the deterministic setting or address almost-sure stochastic requirements. In contrast, we propose a methodology to tackle problems with less stringent requirements than almost sure. To this end, we first establish a connection between two distinct stochastic reach-avoid problems and three classes of stochastic optimal control problems involving discontinuous payoff functions. Subsequently, we focus on solutions of one of the classes of stochastic optimal control problems---the exit-time problem, which solves both the two reach-avoid problems mentioned above. We then derive a weak version of a dynamic programming principle (DPP) for the corresponding value function; in this direction our contribution compared to the existing literature is to develop techniques that admit discontinuous payoff functions. Moreover, based on our DPP, we provide an alternative characterization of the value function as a solution of a partial differential equation in the sense of discontinuous viscosity solutions, along with boundary conditions both in Dirichlet and viscosity senses. Theoretical justifications are also discussed to pave the way for deployment of off-the-shelf PDE solvers for numerical computations. Finally, we validate the performance of the proposed framework on the stochastic Zermelo navigation problem.