A Gradient Descent-Ascent Method for Continuous-Time Risk-Averse Optimal Control

TL;DR

This paper introduces a gradient descent-ascent algorithm for continuous-time stochastic risk-averse optimal control, utilizing duality of coherent risk measures and a min-max reformulation to ensure convergence.

Contribution

It presents a novel explicit algorithm for risk-averse control problems with non-linear dynamics, leveraging duality and smooth reformulation for convergence guarantees.

Findings

Algorithm converges to candidate solutions of the original problem.

Numerical simulations demonstrate efficiency in trajectory tracking.

Risk measures provide advantages over classical expectation in control tasks.

Abstract

In this paper, we consider continuous-time stochastic optimal control problems where the cost is evaluated through a coherent risk measure. We provide an explicit gradient descent-ascent algorithm which applies to problems subject to non-linear stochastic differential equations. More specifically, we leverage duality properties of coherent risk measures to relax the problem via a smooth min-max reformulation which induces artificial strong concavity in the max subproblem. We then formulate necessary conditions of optimality for this relaxed problem which we leverage to prove convergence of the gradient descent-ascent algorithm to candidate solutions of the original problem. Finally, we showcase the efficiency of our algorithm through numerical simulations involving trajectory tracking problems and highlight the benefit of favoring risk measures over classical expectation.