Complexity analysis of regularization methods for implicitly constrained least squares

TL;DR

This paper analyzes the computational complexity of a quadratic regularization method for solving implicitly constrained nonlinear least squares problems, with applications to PDE-constrained optimization, providing new theoretical insights and demonstrating practical efficiency.

Contribution

It introduces a complexity analysis framework for a regularization-based algorithm in implicitly constrained least squares, extending understanding to PDE-constrained problems.

Findings

Quantifies worst-case cost to reach approximate stationary points.

Provides new complexity bounds for implicitly constrained problems.

Demonstrates efficiency through numerical experiments.

Abstract

Optimization problems constrained by partial differential equations (PDEs) naturally arise in scientific computing, as those constraints often model physical systems or the simulation thereof. In an implicitly constrained approach, the constraints are incorporated into the objective through a reduced formulation. To this end, a numerical procedure is typically applied to solve the constraint system, and efficient numerical routines with quantifiable cost have long been developed for that purpose. Meanwhile, the field of complexity in optimization, that estimates the cost of an optimization algorithm, has received significant attention in the literature, with most of the focus being on unconstrained or explicitly constrained problems. In this paper, we analyze an algorithmic framework based on quadratic regularization for implicitly constrained nonlinear least squares. By leveraging adjoint formulations, we can quantify the worst-case cost of our method to reach an approximate stationary point of the optimization problem. Our definition of such points exploits the least-squares structure of the objective, and provides new complexity insights even in the unconstrained setting. Numerical experiments conducted on PDE-constrained optimization problems demonstrate the efficiency of the proposed framework.