Mirror Descent for Deterministic Optimal Control

TL;DR

This paper introduces a mirror descent algorithm for finite-horizon deterministic optimal control, providing convergence guarantees and demonstrating effectiveness through numerical examples.

Contribution

It develops a mirror descent framework inspired by Pontryagin's principle, with theoretical convergence analysis and practical numerical demonstrations.

Findings

Achieves an $O(1/n)$ convergence rate in convex cases.

Attains geometric convergence when control regularization is positive.

Numerical examples include linear-quadratic, degenerate convex, and nonlinear high-dimensional problems.

Abstract

We study an explicit mirror-descent method for finite-horizon deterministic optimal control problems. The method is motivated by Pontryagin's maximum principle: at each iteration, one solves the state and adjoint equations and updates the control by maximizing a first-order approximation of the regularized Hamiltonian penalized by a Bregman divergence. In the Euclidean case, the update reduces to a projected gradient step in the control variable. Under global smoothness assumptions and uniform convexity of the mirror map, we prove a relative smoothness estimate for the cost functional and derive an energy dissipation inequality for sufficiently small step sizes. Under an additional concavity assumption on the unregularized Hamiltonian and convexity of the terminal cost, we establish relative convexity of the regularized objective. These estimates yield an $O(1/n)$ convergence rate in the unregularized convex case and a geometric rate when the control regularization parameter is positive. Numerical examples illustrate the behavior of the method in linear-quadratic, degenerate convex, and nonlinear high-dimensional settings.