Majorization-minimization procedures and convergence of SQP methods for semi-algebraic and tame programs

TL;DR

This paper develops a general convergence framework for majorization-minimization and SQP methods applied to semi-algebraic and tame programs, without requiring second-order or convexity assumptions.

Contribution

It introduces a novel convergence analysis using semi-algebraic geometry and variational analysis, applicable to a broad class of nonlinear programming methods.

Findings

Sequences converge to critical points under broad conditions.

First general convergence results for nonlinear programming without second-order assumptions.

Convergence rates similar to first-order methods.

Abstract

In view of solving nonsmooth and nonconvex problems involving complex constraints (like standard NLP problems), we study general maximization-minimization procedures produced by families of strongly convex sub-problems. Using techniques from semi-algebraic geometry and variational analysis -in particular Lojasiewicz inequality- we establish the convergence of sequences generated by this type of schemes to critical points. The broad applicability of this process is illustrated in the context of NLP. In that case critical points coincide with KKT points. When the data are semi-algebraic or real analytic our method applies (for instance) to the study of various SQP methods: the moving balls method, Sl1QP, ESQP. Under standard qualification conditions, this provides -to the best of our knowledge- the first general convergence results for general nonlinear programming problems. We emphasize the fact that, unlike most works on this subject, no second-order assumption and/or convexity assumptions whatsoever are made. Rate of convergence are shown to be of the same form as those commonly encountered with first order methods.