Differential estimates for fast first-order multilevel nonconvex optimisation

TL;DR

This paper introduces a novel iterative estimation technique for derivatives in multilevel nonconvex optimization, applicable to bilevel and PDE-constrained problems, enhancing convergence analysis and practical performance.

Contribution

It develops a unified single-loop method combining outer optimization updates with inner estimate refinements, extending convergence proofs to inexact and measure space settings.

Findings

Effective in Electrical Impedance Tomography applications

Provides improved convergence results for nonconvex primal-dual methods

Demonstrates practical benefits in PDE-constrained optimization

Abstract

With a view on bilevel and PDE-constrained optimisation, we develop iterative estimates $\widetilde{F'}(x^k)$ of $F'(x^k)$ for composite functions $F :=J \circ S$, where $S$ is the solution mapping of the inner optimisation problem or PDE. The idea is to form a single-loop method by interweaving updates of the iterate $x^k$ by an outer optimisation method, with updates of the estimate by single steps of standard optimisation methods and linear system solvers. When the inner methods satisfy simple tracking inequalities, the differential estimates can almost directly be employed in standard convergence proofs for general forward-backward type methods. We adapt those proofs to a general inexact setting in normed spaces, that, besides our differential estimates, also covers mismatched adjoints and unreachable optimality conditions in measure spaces. As a side product of these efforts, we provide improved convergence results for nonconvex Primal-Dual Proximal Splitting (PDPS). We numerically evaluate our methods on Electrical Impedance Tomography (EIT) and minimal surface control.