Solving Mission-Wide Chance-Constrained Optimal Control Using Dynamic Programming

TL;DR

This paper develops a dynamic programming approach for solving mission-wide chance-constrained optimal control problems, addressing the challenge of correlated constraints over entire trajectories by state augmentation.

Contribution

It introduces a novel DP solution for MWCC-OCP by transforming the problem with state augmentation, overcoming limitations of stage-wise chance constraint methods.

Findings

Provides conditions for classical DP applicability to MWCC-OCP.

Proposes a state augmentation technique to enable DP solution.

Addresses the correlation issue in mission-wide chance constraints.

Abstract

This paper aims to provide a Dynamic Programming (DP) approach to solve the Mission-Wide Chance-Constrained Optimal Control Problems (MWCC-OCP). The mission-wide chance constraint guarantees that the probability that the entire state trajectory lies within a constraint/safe region is higher than a prescribed level, and is different from the stage-wise chance constraints imposed at individual time steps. The control objective is to find an optimal policy sequence that achieves both (i) satisfaction of a mission-wide chance constraint, and (ii) minimization of a cost function. By transforming the stage-wise chance-constrained problem into an unconstrained counterpart via Lagrangian method, standard DP can then be deployed. Yet, for MWCC-OCP, this methods fails to apply, because the mission-wide chance constraint cannot be easily formulated using stage-wise chance constraints due to the time-correlation between the latter (individual states are coupled through the system dynamics). To fill this gap, firstly, we detail the conditions required for a classical DP solution to exist for this type of problem; secondly, we propose a DP solution to the MWCC-OCP through state augmentation by introducing an additional functional state variable.