Penalty methods to compute stationary solutions in constrained optimization problems

TL;DR

This paper investigates the relationship between stationary solutions of penalized unconstrained problems and original constrained problems, introducing an algorithm for approximate solutions with convergence guarantees even without Clarke regularity.

Contribution

It establishes connections between stationary points of penalized and original problems, introduces an approximate stationarity concept, and proposes an algorithm with a convergence rate of O(ε^{-2}).

Findings

Stationary solutions of penalized problems relate to original constrained solutions.

Proposed algorithm computes approximate stationary solutions with O(ε^{-2}) rate.

Characterization of the semi-differentiability of the distance function based on set geometry.

Abstract

This paper is devoted to studying the stationary solutions of a general constrained optimization problem through its associated unconstrained penalized problems. We aim to answer the question, "what do the stationary solutions of a penalized unconstrained problem tell us about the solutions of the original constrained optimization problem?". We answer the latter question by establishing relationships between global (local) minimizers and stationary points of the two optimization problems. Given the strong connection between stationary solutions between problems, we introduce a new approximate $\varepsilon$-stationary solution for the constrained optimization problems. We propose an algorithm to compute such an approximate stationary solution for a general constrained optimization problem, even in the absence of Clarke regularity. Under reasonable assumptions, we establish the rate $O(\varepsilon^{-2})$ for our algorithm, akin to the gradient descent method for smooth minimization. Since our penalty terms are constructed by the powers of the distance function, our stationarity analysis heavily depends on the generalized differentiation of the distance function. In particular, we characterize the (semi-)differentiability of the distance function $\mbox{dist}(. ;X)$ defined by a Fr\'echet smooth norm, in terms of the geometry of the set $X$. We show that $\mbox{dist} (. ;X)$ is semi-differentiable if and only if $X$ is geometrically derivable. The latter opens the door to design optimization algorithms for constrained optimization problems that suffer from Clarke irregularity in their objectives and constraint functions.