Generalized Metric Subregularity with Applications to High-Order Regularized Newton Methods

TL;DR

This paper introduces a new concept called generalized metric subregularity, extending traditional conditions, and applies it to develop high-order Newton methods with superlinear and quadratic convergence guarantees.

Contribution

It develops the theory of generalized metric subregularity, provides verifiable conditions, and designs a high-order Newton method with proven convergence rates.

Findings

Superlinear convergence under generalized metric subregularity

Quadratic convergence with additional assumptions

Global convergence in over-parameterized compressed sensing

Abstract

This paper pursues a twofold goal. First, we introduce and study in detail a new notion of variational analysis called generalized metric subregularity, which is a far-going extension of the conventional metric subregularity conditions. Our primary focus is on examining this concept concerning first-order and second-order stationary points. We develop an extended convergence framework that enables us to derive superlinear and quadratic convergence under the generalized metric subregularity condition, broadening the widely used KL convergence analysis framework. We present verifiable sufficient conditions to ensure the proposed generalized metric subregularity condition and provide examples demonstrating that the derived convergence rates are sharp. Second, we design a new high-order regularized Newton method with momentum steps, and apply the generalized metric subregularity to establish its superlinear convergence. Quadratic convergence is obtained under additional assumptions. Specifically, when applying the proposed method to solve the (nonconvex) over-parameterized compressed sensing model, we achieve global convergence with a quadratic local convergence rate towards a global minimizer under a strict complementarity condition.