An Inexact General Descent Method with Applications in Differential Equation-Constrained Optimization

TL;DR

This paper introduces an inexact descent optimization framework suitable for differential equation-constrained problems, demonstrating improved efficiency through adaptive gradient accuracy in practical inverse problems.

Contribution

It develops a new convergence theory for inexact gradient methods with adaptive evaluation, specifically tailored for DECO applications, and implements practical algorithms with demonstrated efficiency gains.

Findings

Adaptive inexact gradients reduce optimization time.

Inexact BFGS-like method improves convergence.

Adaptive accuracy enhances efficiency in inverse problems.

Abstract

In many applications, gradient evaluations are inherently approximate, motivating the development of optimization methods that remain reliable under inexact first-order information. A common strategy in this context is adaptive evaluation, whereby coarse gradients are used in early iterations and refined near a minimizer. This is particularly relevant in differential equation-constrained optimization (DECO), where discrete adjoint gradients depend on iterative solvers. Motivated by DECO applications, we propose an inexact general descent framework and establish its global convergence theory under two step-size regimes. For bounded step sizes, the analysis assumes that the error tolerance in the computed gradient is proportional to its norm, whereas for diminishing step sizes, the tolerance sequence is required to be summable. The framework is implemented through inexact gradient descent and an inexact BFGS-like method, whose performance is demonstrated on a second-order ODE inverse problem and a two-dimensional Laplace inverse problem using discrete adjoint gradients with adaptive accuracy. Across these examples, adaptive inexact gradients consistently reduced optimization time relative to fixed tight tolerances, while incorporating curvature information further improved overall efficiency.