Efficient quadratic penalization through the partial minimization technique

TL;DR

This paper demonstrates that partial minimization regularizes quadratically penalized problems, improving conditioning and enabling efficient, robust solutions for complex parameter estimation and PDE-constrained optimization tasks.

Contribution

It introduces a novel perspective on partial minimization as a regularization technique, enhancing variable projection and penalty methods for better computational stability.

Findings

Partial minimization improves problem conditioning.

The method allows loose solutions in iterative subproblems.

Applications include boundary control and optimal transport.

Abstract

Common computational problems, such as parameter estimation in dynamic models and PDE constrained optimization, require data fitting over a set of auxiliary parameters subject to physical constraints over an underlying state. Naive quadratically penalized formulations, commonly used in practice, suffer from inherent ill-conditioning. We show that surprisingly the partial minimization technique regularizes the problem, making it well-conditioned. This viewpoint sheds new light on variable projection techniques, as well as the penalty method for PDE constrained optimization, and motivates robust extensions. In addition, we outline an inexact analysis, showing that the partial minimization subproblem can be solved very loosely in each iteration. We illustrate the theory and algorithms on boundary control, optimal transport, and parameter estimation for robust dynamic inference.