Online Adjoint Methods for Optimization of PDEs

TL;DR

This paper introduces and analyzes an online adjoint algorithm for PDE optimization that updates design variables continuously, offering a potentially more efficient alternative to traditional methods by ensuring convergence to optimal solutions.

Contribution

The paper provides a rigorous mathematical analysis of the convergence and rate of an online adjoint algorithm for PDE optimization, extending the pseudo-time-stepping approach.

Findings

The online adjoint algorithm asymptotically converges to the steepest descent direction.

Under certain conditions, the algorithm converges to a critical point of the objective.

A multi-scale analysis is key to proving convergence and rates.

Abstract

We present and mathematically analyze an online adjoint algorithm for the optimization of partial differential equations (PDEs). Traditional adjoint algorithms would typically solve a new adjoint PDE at each optimization iteration, which can be computationally costly. In contrast, an online adjoint algorithm updates the design variables in continuous-time and thus constantly makes progress towards minimizing the objective function. The online adjoint algorithm we consider is similar in spirit to the the pseudo-time-stepping, one-shot method which has been previously proposed. Motivated by the application of such methods to engineering problems, we mathematically study the convergence of the online adjoint algorithm. The online adjoint algorithm relies upon a time-relaxed adjoint PDE which provides an estimate of the direction of steepest descent. The algorithm updates this estimate continuously in time, and it asymptotically converges to the exact direction of steepest descent as $t \rightarrow \infty$. We rigorously prove that the online adjoint algorithm converges to a critical point of the objective function for optimizing the PDE. Under appropriate technical conditions, we also prove a convergence rate for the algorithm. A crucial step in the convergence proof is a multi-scale analysis of the coupled system for the forward PDE, adjoint PDE, and the gradient descent ODE for the design variables.