Multilevel Stochastic Gradient Descent for Optimal Control Under Uncertainty

TL;DR

This paper introduces a multilevel stochastic gradient descent method that efficiently solves high-dimensional optimal control problems governed by PDEs under uncertainty, leveraging parallel multilevel Monte Carlo techniques.

Contribution

The paper develops a novel multilevel stochastic gradient descent approach combining multilevel Monte Carlo with PDE-constrained control, improving convergence and computational efficiency.

Findings

Linear convergence in optimization steps

Outperforms standard stochastic gradient descent in speed and accuracy

Effective for high-dimensional control problems

Abstract

We present a multilevel stochastic gradient descent method for the optimal control of systems governed by partial differential equations under uncertain input data. The gradient descent method used to find the optimal control leverages a parallel multilevel Monte Carlo method as stochastic gradient estimator. As a result, we achieve precise control over the stochastic gradient's bias, introduced by numerical approximation, and its sampling error, arising from the use of incomplete gradients, while optimally managing computational resources. We show that the method exhibits linear convergence in the number of optimization steps while avoiding the cost of computing the full gradient at the highest fidelity. Numerical experiments demonstrate that the method significantly outperforms the standard (mini-) batched stochastic gradient descent method in terms of convergence speed and accuracy. The method is particularly well-suited for high-dimensional control problems, taking advantage of parallel computing resources and a distributed multilevel data structure. Additionally, we evaluate and implement different step size strategies, optimizer schemes, and budgeting techniques. The method's performance is studied using a two-dimensional elliptic subsurface diffusion problem with log-normal coefficients and Mat\'ern covariance.