State Constrained Stochastic Optimal Control Using LSTMs

TL;DR

This paper introduces a novel LSTM-based deep learning approach for solving state constrained stochastic optimal control problems, effectively handling high dimensionality, non-linearities, and long horizons.

Contribution

It develops a new methodology combining FBSDE solutions with LSTM networks and a penalty-based constraint handling scheme, improving scalability and robustness.

Findings

Effective handling of state constraints via penalty functions.

Demonstrated scalability on high-dimensional control problems.

Numerical results show improved stability and performance.

Abstract

In this paper, we propose a new methodology for state constrained stochastic optimal control (SOC) problems. The solution is based on past work in solving SOC problems using forward-backward stochastic differential equations (FBSDE). Our approach in solving the FBSDE utilizes a deep neural network (DNN), specifically Long Short-Term Memory (LSTM) networks. LSTMs are chosen to solve the FBSDE to address the curse of dimensionality, non-linearities, and long time horizons. In addition, the state constraints are incorporated using a hard penalty function, resulting in a controller that respects the constraint boundaries. Numerical instability that would be introduced by the penalty function is dealt with through an adaptive update scheme. The control design methodology is applicable to a large class of control problems. The performance and scalability of our proposed algorithm are demonstrated by numerical simulations.