On Convergence Lemma and Convergence Stability for Piecewise Analytic Functions

TL;DR

This paper establishes a convergence lemma for compositions of analytic functions and the maximum operator, introduces a geometric characterization of stationary points, and proposes a new concept of convergence stability for nonsmooth nonconvex optimization.

Contribution

It proves a convergence lemma specific to analytic compositions, extends geometric analysis of stationary points, and introduces the notion of convergence stability in nonsmooth nonconvex optimization.

Findings

Convergence points near a local minimum shrink as tolerance decreases.

Analyticity is essential for the convergence lemma's validity.

A geometric characterization of stationary points is provided.

Abstract

In this work, a convergence lemma for function $f$ being finite compositions of analytic mappings and the maximum operator is proved. The lemma shows that the set of $\delta$-stationary points near an isolated local minimum point $x^*$ is shrinking to $x^*$ as $\delta\to 0$. It is a natural extension of the version for strongly convex $C^1$ functions. However, the correctness of the lemma is subtle. Analytic mappings are necessary for the lemma in the sense that replacing it with differentiable or $C^\infty$ mappings makes the lemma false. The proof is based on stratification theorems of semi-analytic sets by {\L}ojasiewicz. An extension of this proof presents a geometric characterization of the set of stationary points of $f$. Finally, a notion of stability on stationary points, called convergence stability, is proposed. It asks, under small numerical errors, whether a reasonable convergent optimization method started near a stationary point should eventually converge to the same stationary point. The concept of convergence stability becomes nontrivial qualitatively only when the objective function is both nonsmooth and nonconvex. Via the convergence lemma, an intuitive equivalent condition for convergence stability of $f$ is proved. These results together provide a new geometric perspective to study the problem of "where-to-converge" in nonsmooth nonconvex optimization.