Ghost Penalties in Nonconvex Constrained Optimization: Diminishing Stepsizes and Iteration Complexity

TL;DR

This paper introduces a novel convergence analysis framework for nonconvex constrained optimization that uses penalty functions only in theory, enabling new results on diminishing stepsize methods and SQP algorithm complexities.

Contribution

It presents the first general convergence analysis for diminishing stepsize methods in nonconvex constrained optimization, using penalty functions solely for theoretical purposes.

Findings

Established convergence to generalized stationary points.

Provided complexity bounds for SQP-type algorithms.

Introduced penalty functions as a theoretical tool rather than an algorithm component.

Abstract

We consider nonconvex constrained optimization problems and propose a new approach to the convergence analysis based on penalty functions. We make use of classical penalty functions in an unconventional way, in that penalty functions only enter in the theoretical analysis of convergence while the algorithm itself is penalty-free. Based on this idea, we are able to establish several new results, including the first general analysis for diminishing stepsize methods in nonconvex, constrained optimization, showing convergence to generalized stationary points, and a complexity study for SQP-type algorithms.