Computing Optimal Joint Chance Constrained Control Policies

TL;DR

This paper develops a dynamic programming approach to compute optimal control policies for stochastic Markovian systems with joint chance constraints, overcoming the limitations of standard methods.

Contribution

It introduces a novel augmentation of system dynamics with a binary state to enable optimal policy computation under joint chance constraints.

Findings

Proposes a new dynamic programming method for joint chance constrained control.

Characterizes the structure of optimal policies in stochastic systems.

Provides a computational framework for provably optimal solutions.

Abstract

We consider the problem of optimally controlling stochastic, Markovian systems subject to joint chance constraints over a finite-time horizon. For such problems, standard Dynamic Programming is inapplicable due to the time correlation of the joint chance constraints, which calls for non-Markovian, and possibly stochastic, policies. Hence, despite the popularity of this problem, solution approaches capable of providing provably-optimal and easy-to-compute policies are still missing. We fill this gap by augmenting the dynamics via a binary state, allowing us to characterize the optimal policies and develop a Dynamic Programming based solution method.