Learning Gradient Flow: Using Equation Discovery to Accelerate Engineering Optimization

TL;DR

This paper introduces Learned Gradient Flow (LGF), a data-driven method that models and accelerates optimization processes by learning continuous-time gradient dynamics from trajectory data, reducing computational costs.

Contribution

The paper presents a novel approach to use equation discovery for modeling gradient flows, enabling surrogate optimization that speeds up convergence in complex problems.

Findings

Learned gradient flows improve convergence speed.

Surrogate models reduce evaluation costs.

Effective on engineering and scientific optimization problems.

Abstract

In this work, we investigate the use of data-driven equation discovery for dynamical systems to model and forecast continuous-time dynamics of unconstrained optimization problems. To avoid expensive evaluations of the objective function and its gradient, we leverage trajectory data on the optimization variables to learn the continuous-time dynamics associated with gradient descent, Newton's method, and ADAM optimization. The discovered gradient flows are then solved as a surrogate for the original optimization problem. To this end, we introduce the Learned Gradient Flow (LGF) optimizer, which is equipped to build surrogate models of variable polynomial order in full- or reduced-dimensional spaces at user-defined intervals in the optimization process. We demonstrate the efficacy of this approach on several standard problems from engineering mechanics and scientific machine learning, including two inverse problems, structural topology optimization, and two forward solves with different discretizations. Our results suggest that the learned gradient flows can significantly expedite convergence by capturing critical features of the optimization trajectory while avoiding expensive evaluations of the objective and its gradient.