A Generalized Newton Method for Subgradient Systems

TL;DR

This paper introduces a new Newton-type algorithm for solving subgradient systems of prox-regular functions, with proven local superlinear convergence and applications in machine learning and statistics.

Contribution

It develops a generalized Newton method based on second-order subdifferentials for broad classes of functions, extending existing algorithms and providing verifiable convergence conditions.

Findings

Algorithm achieves local superlinear convergence.

Applicable to ${ m C}^{1,1}$ functions and structured composite functions.

Demonstrated effectiveness through examples and applications in machine learning.

Abstract

This paper proposes and develops a new Newton-type algorithm to solve subdifferential inclusions defined by subgradients of extended-real-valued prox-regular functions. The proposed algorithm is formulated in terms of the second-order subdifferential of such functions that enjoys extensive calculus rules and can be efficiently computed for broad classes of extended-real-valued functions. Based on this and on metric regularity and subregularity properties of subgradient mappings, we establish verifiable conditions ensuring well-posedness of the proposed algorithm and its local superlinear convergence. The obtained results are also new for the class of equations defined by continuously differentiable functions with Lipschitzian gradients (${\cal C}^{1,1}$ functions), which is the underlying case of our consideration. The developed algorithms for prox-regular functions and its extension to a structured class of composite functions are formulated in terms of proximal mappings and forward-backward envelopes. Besides numerous illustrative examples and comparison with known algorithms for ${\cal C}^{1,1}$ functions and generalized equations, the paper presents applications of the proposed algorithms to regularized least square problems arising in statistics, machine learning, and related disciplines.